Visual Story Generation Based on Emotion and Keywords

Yuetian Chen, Ruohua Li, Bowen Shi, Peiru Liu, Mei Si
1/7/2023
cs.AIcs.CLcs.LG

No official code found. Best implementation: Stry233/Visual-Story-Generation-Based-on-Emotional-and-Keyword-Scheme (7 stars, MIT, last commit 2024-03-05).

Abstract

Automated visual story generation aims to produce stories with corresponding illustrations that exhibit coherence, progression, and adherence to characters' emotional development. This work proposes a story generation pipeline to co-create visual stories with the users. The pipeline allows the user to control events and emotions on the generated content. The pipeline includes two parts: narrative and image generation. For narrative generation, the system generates the next sentence using user-specified keywords and emotion labels. For image generation, diffusion models are used to create a visually appealing image corresponding to each generated sentence. Further, object recognition is applied to the generated images to allow objects in these images to be mentioned in future story development.

View on arXivView PDF

Code Implementations(1)

Stry233/Visual-Story-Generation-Based-on-Emotional-and-Keyword-SchemeCommunity100%
MIT

We designed an end-to-end framework that encourage interactive narrative experience based on keyword control and image generation.

73CSS, HTMLAug 11, 20222 years agoMIT
emotion-detectionlanguage-modelstorytellingvisual-storytelling

Similar Papers

Imagining from Images with an AI Storytelling Tool

84% match2024

EmoStory: Emotion-Aware Story Generation

84% match2026

Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences

82% match1 repo72023

SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling

81% match2024

DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion

81% match2024

Visual Storytelling with Question-Answer Plans

80% match2023

Cite this paper

@article{chen2023visual,
  title  = {Visual Story Generation Based on Emotion and Keywords},
  author = {Yuetian Chen and Ruohua Li and Bowen Shi and Peiru Liu and Mei Si},
  year   = {2023},
  eprint = {2301.02777},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.02777v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.02777.svg)](https://codeofpaper.com/paper/2301.02777)

Discussion