StyleDrop: Text-to-Image Generation in Any Style

Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok +9 more
6/1/2023
cs.CVcs.AI

No official code found. Best implementation: zideliu/StyleDrop-PyTorch (587 stars, MIT, last commit 2023-08-23). 1 more below.

Abstract

Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts. However, ambiguities inherent in natural language and out-of-distribution effects make it hard to synthesize image styles, that leverage a specific design pattern, texture or material. In this paper, we introduce StyleDrop, a method that enables the synthesis of images that faithfully follow a specific style using a text-to-image model. The proposed method is extremely versatile and captures nuances and details of a user-provided style, such as color schemes, shading, design patterns, and local and global effects. It efficiently learns a new style by fine-tuning very few trainable parameters (less than $1\%$ of total model parameters) and improving the quality via iterative training with either human or automated feedback. Better yet, StyleDrop is able to deliver impressive results even when the user supplies only a single image that specifies the desired style. An extensive study shows that, for the task of style tuning text-to-image models, StyleDrop implemented on Muse convincingly outperforms other methods, including DreamBooth and textual inversion on Imagen or Stable Diffusion. More results are available at our project website: https://styledrop.github.io

View on arXivView PDF

Code Implementations(2)

zideliu/StyleDrop-PyTorchCommunity64%
MIT

Unoffical implement for [StyleDrop](https://arxiv.org/abs/2306.00983)

58730Python, Jupyter NotebookJul 4, 20233 years agoMIT
aim-uofa/StyleDrop-PyTorchCommunity63%
MIT

This is an unofficial PyTorch implementation of StyleDrop: Text-to-Image Generation in Any Style.

22615Python, Jupyter NotebookJul 11, 20233 years agoMIT

Similar Papers

StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models

89% match1 repo102025

MuseumMaker: Continual Style Customization without Catastrophic Forgetting

89% match2024

StyleInject: Parameter Efficient Tuning of Text-to-Image Diffusion Models

88% match2024

A Novel Sampling Scheme for Text- and Image-Conditional Image Synthesis in Quantized Latent Spaces

88% match1 repo7482022

DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models

88% match1 repo542023

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

88% match2023

Cite this paper

@article{sohn2023styledrop,
  title  = {StyleDrop: Text-to-Image Generation in Any Style},
  author = {Kihyuk Sohn and Nataniel Ruiz and Kimin Lee and Daniel Castro Chin and Irina Blok and Huiwen Chang and Jarred Barber and Lu Jiang and Glenn Entis and Yuanzhen Li and Yuan Hao and Irfan Essa and Michael Rubinstein and Dilip Krishnan},
  year   = {2023},
  eprint = {2306.00983},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.00983v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.00983.svg)](https://codeofpaper.com/paper/2306.00983)

Discussion