Zero-Shot Text-to-Image Generation

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss +3 more
2/24/2021
cs.CVcs.LG

Official code: openai/DALL-E (10,816 stars, last commit 2024-01-31).

Abstract

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset. These assumptions might involve complex architectures, auxiliary losses, or side information such as object part labels or segmentation masks supplied during training. We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

View on arXivView PDF

Code Implementations(1)

openai/DALL-EOfficial100%

PyTorch package for the discrete VAE used for DALL·E.

10,8161,848Python, Jupyter NotebookFeb 23, 20212 years agoNOASSERTION

Similar Papers

Lformer: Text-to-Image Generation with L-shape Block Parallel Decoding

87% match2023

Randomized Autoregressive Visual Generation

86% match1 repo1.2k2024

Next Patch Prediction for Autoregressive Visual Generation

86% match1 repo1292024

A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generation

85% match1 repo812024

Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis

84% match1 repo2052024

Frequency Autoregressive Image Generation with Continuous Tokens

84% match1 repo1012025

Cite this paper

@article{ramesh2021zeroshot,
  title  = {Zero-Shot Text-to-Image Generation},
  author = {Aditya Ramesh and Mikhail Pavlov and Gabriel Goh and Scott Gray and Chelsea Voss and Alec Radford and Mark Chen and Ilya Sutskever},
  year   = {2021},
  eprint = {2102.12092},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2102.12092}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2102.12092.svg)](https://codeofpaper.com/paper/2102.12092)

Discussion