DreamDiffusion: Generating High-Quality Images from Brain EEG Signals

Yunpeng Bai, Xintao Wang, Yan-pei Cao, Yixiao Ge, Chun Yuan +1 more
6/29/2023
cs.CV

Official code: bbaaii/DreamDiffusion (660 stars, MIT, last commit 2025-10-01). 2 more implementations below.

Abstract

This paper introduces DreamDiffusion, a novel method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate thoughts into text. DreamDiffusion leverages pre-trained text-to-image models and employs temporal masked signal modeling to pre-train the EEG encoder for effective and robust EEG representations. Additionally, the method further leverages the CLIP image encoder to provide extra supervision to better align EEG, text, and image embeddings with limited EEG-image pairs. Overall, the proposed method overcomes the challenges of using EEG signals for image generation, such as noise, limited information, and individual differences, and achieves promising results. Quantitative and qualitative results demonstrate the effectiveness of the proposed method as a significant step towards portable and low-cost ``thoughts-to-image'', with potential applications in neuroscience and computer vision. The code is available here \url{https://github.com/bbaaii/DreamDiffusion}.

View on arXivView PDF

Code Implementations(3)

bbaaii/DreamDiffusionOfficial100%
MIT

Implementation of “DreamDiffusion: Generating High-Quality Images from Brain EEG Signals”

66080PythonJun 30, 20231 years agoMIT
Jvinodtrmble/DreamDiffusionCommunity74%
MIT

Implementation of “DreamDiffusion: Generating High-Quality Images from Brain EEG Signals”

00PythonJul 7, 20251 years agoMIT
dan-crdll/nn_project_dreamdiffusionCommunity68%
MIT

Re-implementation of the method proposed in ''DreamDiffusion: Generating High-Quality Images from Brain EEG Signals'' by Y. Bai, X. Wang et al. for Neural Network Course exam Topics

100Python, Jupyter NotebookMay 28, 20242 years agoMIT
bcidiffusion-modelseegmachine-learningmae+1 more

Similar Papers

BrainDreamer: Reasoning-Coherent and Controllable Image Generation from EEG Brain Signals via Language Guidance

89% match2024

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion

87% match2 repos2202024

BrainDecoder: Style-Based Visual Decoding of EEG Signals

86% match2024

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

85% match2023

CATVis: Context-Aware Thought Visualization

85% match2025

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

84% match2026

Cite this paper

@article{bai2023dreamdiffusion,
  title  = {DreamDiffusion: Generating High-Quality Images from Brain EEG Signals},
  author = {Yunpeng Bai and Xintao Wang and Yan-pei Cao and Yixiao Ge and Chun Yuan and Ying Shan},
  year   = {2023},
  eprint = {2306.16934},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.16934v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.16934.svg)](https://codeofpaper.com/paper/2306.16934)

Discussion