DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion

Johanna Karras, Aleksander Holynski, Ting-Chun Wang, Ira Kemelmacher-Shlizerman
4/1/2023

No official code found. Best implementation: johannakarras/DreamPose (1,006 stars, MIT, last commit 2023-11-02).

Abstract

We present DreamPose, a diffusion-based method for generating animated fashion videos from still images. Given an image and a sequence of human body poses, our method synthesizes a video containing both human and fabric motion. To achieve this, we transform a pre-trained text-to-image model (Stable Diffusion [16]) into a pose-and-image guided video synthesis model, using a novel finetuning strategy, a set of architectural changes to support the added conditioning signals, and techniques to encourage temporal consistency. We fine-tune on a collection of fashion videos from the UBC Fashion dataset [50]. We evaluate our method on a variety of clothing styles and poses, and demonstrate that our method produces state-of-the-art results on fashion video animation. Video results are available on our project page: https://grail.cs.washington.edu/projects/dreampose

View on arXivView PDF

Code Implementations(1)

johannakarras/DreamPoseCommunity70%
MIT

Official implementation of "DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion"

1,00678PythonMar 21, 20232 years agoMIT

Similar Papers

DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion

99% match2 repos1.0k2023

FashionFlow: Leveraging Diffusion Models for Dynamic Fashion Video Synthesis from Static Imagery

92% match1 repo52023

WildVidFit: Video Virtual Try-On in the Wild via Image-Based Controlled Diffusion Models

88% match2024

FloAt: Flow Warping of Self-Attention for Clothing Animation Generation

88% match2024

DANCER: Dance ANimation via Condition Enhancement and Rendering with diffusion model

86% match2025

ViViD: Video Virtual Try-on using Diffusion Models

86% match1 repo5682024

Cite this paper

@article{karras2023dreampose,
  title  = {DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion},
  author = {Johanna Karras and Aleksander Holynski and Ting-Chun Wang and Ira Kemelmacher-Shlizerman},
  year   = {2023},
  eprint = {2304.06025},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2304.06025},
  journal = {ICCV 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.06025.svg)](https://codeofpaper.com/paper/2304.06025)

Discussion