Brain2Music: Reconstructing Music from Human Brain Activity

Timo I. Denk, Yu Takagi, Takuya Matsuyama, Andrea Agostinelli, Tomoya Nakai +2 more
7/20/2023
q-bio.NCcs.LGcs.SD

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

The process of reconstructing experiences from human brain activity offers a unique lens into how the brain interprets and represents the world. In this paper, we introduce a method for reconstructing music from brain activity, captured using functional magnetic resonance imaging (fMRI). Our approach uses either music retrieval or the MusicLM music generation model conditioned on embeddings derived from fMRI data. The generated music resembles the musical stimuli that human subjects experienced, with respect to semantic properties like genre, instrumentation, and mood. We investigate the relationship between different components of MusicLM and brain activity through a voxel-wise encoding modeling analysis. Furthermore, we discuss which brain regions represent information derived from purely textual descriptions of music stimuli. We provide supplementary material including examples of the reconstructed music at https://google-research.github.io/seanet/brain2music

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

R&B -- Rhythm and Brain: Cross-subject Decoding of Music from Human Brain Activity

88% match1 repo22024

Predicting Artificial Neural Network Representations to Learn Recognition Model for Music Identification from Brain Recordings

82% match1 repo32024

Naturalistic Music Decoding from EEG Data via Latent Diffusion Models

81% match2024

Simulation of Neural Responses to Classical Music Using Organoid Intelligence Methods

81% match2 repos112024

Decoding Selective Auditory Attention to Musical Elements in Ecologically Valid Music Listening

81% match2025

Better audio representations are more brain-like: linking model-brain alignment with performance in downstream auditory tasks

80% match2025

Cite this paper

@article{denk2023brainmusic,
  title  = {Brain2Music: Reconstructing Music from Human Brain Activity},
  author = {Timo I. Denk and Yu Takagi and Takuya Matsuyama and Andrea Agostinelli and Tomoya Nakai and Christian Frank and Shinji Nishimoto},
  year   = {2023},
  eprint = {2307.11078},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.11078v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.11078.svg)](https://codeofpaper.com/paper/2307.11078)

Discussion