Generating Images from Captions with Attention

Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov
11/9/2015
cs.LGcs.CV

Official code: mansimov/text2image (590 stars, MIT, last commit 2017-02-08).

Abstract

Motivated by the recent progress in generative models, we introduce a model that generates images from natural language descriptions. The proposed model iteratively draws patches on a canvas, while attending to the relevant words in the description. After training on Microsoft COCO, we compare our model with several baseline generative models on image generation and retrieval tasks. We demonstrate that our model produces higher quality samples than other approaches and generates images with novel scene compositions corresponding to previously unseen captions in the dataset.

View on arXivView PDF

Code Implementations(1)

mansimov/text2imageOfficial100%
MIT

Generating Images from Captions with Attention

590121PythonJan 2, 20169 years agoMIT

Similar Papers

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators

88% match1 repo1.2k2021

Can Shape-Infused Joint Embeddings Improve Image-Conditioned 3D Diffusion?

85% match2024

Patch-Based 3D Natural Scene Generation from a Single Example

85% match2023

3DGEN: A GAN-based approach for generating novel 3D models from image data

85% match2023

Chimera: Compositional Image Generation using Part-based Concepting

85% match2025

Text2Scene: Generating Compositional Scenes from Textual Descriptions

85% match1 repo1182018

Cite this paper

@article{mansimov2015generating,
  title  = {Generating Images from Captions with Attention},
  author = {Elman Mansimov and Emilio Parisotto and Jimmy Lei Ba and Ruslan Salakhutdinov},
  year   = {2015},
  eprint = {1511.02793},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1511.02793}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1511.02793.svg)](https://codeofpaper.com/paper/1511.02793)

Discussion