Stochastic Video Generation with a Learned Prior

Remi Denton, Rob Fergus
2/21/2018
cs.CVcs.AIcs.LG

Official code: edenton/svg (188 stars, last commit 2023-04-05).

Abstract

Generating video frames that accurately predict future world states is challenging. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce an unsupervised video generation model that learns a prior model of uncertainty in a given environment. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datasets. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

View on arXivView PDF

Code Implementations(1)

edenton/svgOfficial100%
18856Lua, ShellFeb 15, 20183 years ago

Similar Papers

EVA: An Embodied World Model for Future Video Anticipation

86% match1 repo62024

Video Diffusion Models with Local-Global Context Guidance

86% match1 repo382023

Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipation

85% match1 repo242024

Stochastic Adversarial Video Prediction

84% match1 repo3042018

STDiff: Spatio-temporal Diffusion for Continuous Stochastic Video Prediction

84% match1 repo312023

VChain: Chain-of-Visual-Thought for Reasoning in Video Generation

83% match1 repo1202025

Cite this paper

@article{denton2018stochastic,
  title  = {Stochastic Video Generation with a Learned Prior},
  author = {Remi Denton and Rob Fergus},
  year   = {2018},
  eprint = {1802.07687},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1802.07687}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1802.07687.svg)](https://codeofpaper.com/paper/1802.07687)

Discussion