ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG

Xiran Xu, Bo Wang, Yujie Yan, Haolin Zhu, Zechen Zhang +2 more
1/10/2024
eess.SPcs.LG

Official code: xuxiran/ConvConcatNet (3 stars, last commit 2026-04-25).

Abstract

To investigate the processing of speech in the brain, simple linear models are commonly used to establish a relationship between brain signals and speech features. However, these linear models are ill-equipped to model a highly dynamic and complex non-linear system like the brain. Although non-linear methods with neural networks have been developed recently, reconstructing unseen stimuli from unseen subjects' EEG is still a highly challenging task. This work presents a novel method, ConvConcatNet, to reconstruct mel-specgrams from EEG, in which the deep convolution neural network and extensive concatenation operation were combined. With our ConvConcatNet model, the Pearson correlation between the reconstructed and the target mel-spectrogram can achieve 0.0420, which was ranked as No.1 in the Task 2 of the Auditory EEG Challenge. The codes and models to implement our work will be available on Github: https://github.com/xuxiran/ConvConcatNet

View on arXivView PDF

Code Implementations(1)

xuxiran/ConvConcatNetOfficial100%

Deep Convolutional Neural Network, Concat, Auditory EEG Challenge - ICASSP 2024

30PythonDec 30, 20236 months ago

Similar Papers

EEGDiR: Electroencephalogram denoising network for temporal information storage and global modeling through Retentive Network

82% match1 repo72024

Decoding non-invasive brain activity with novel deep-learning approaches

82% match2025

EEG2Mel: Reconstructing Sound from Brain Responses to Music

81% match1 repo182022

EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces

81% match1 repo1.6k2016

Consumer-friendly EEG-based Emotion Recognition System: A Multi-scale Convolutional Neural Network Approach

81% match2025

A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond

81% match2025

Cite this paper

@article{xu2024convconcatnet,
  title  = {ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG},
  author = {Xiran Xu and Bo Wang and Yujie Yan and Haolin Zhu and Zechen Zhang and Xihong Wu and Jing Chen},
  year   = {2024},
  eprint = {2401.04965},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.04965v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.04965.svg)](https://codeofpaper.com/paper/2401.04965)

Discussion