AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Robust Phonocardiogram Classification

Md. Saiful Bari Siddiqui, Utsab Saha
9/27/2025
eess.AScs.AIcs.LG

No official code found. Best implementation: Saiful185/AudioFuse (0 stars, MIT, last commit 2026-04-23).

Abstract

Biomedical audio signals, such as phonocardiograms (PCG), are inherently rhythmic and contain diagnostic information in both their spectral (tonal) and temporal domains. Standard 2D spectrograms provide rich spectral features but compromise the phase information and temporal precision of the 1D waveform. We propose AudioFuse, an architecture that simultaneously learns from both complementary representations to classify PCGs. To mitigate the overfitting risk common in fusion models, we integrate a custom, wide-and-shallow Vision Transformer (ViT) for spectrograms with a shallow 1D CNN for raw waveforms. On the PhysioNet 2016 dataset, AudioFuse achieves a state-of-the-art competitive ROC-AUC of 0.8608 when trained from scratch, outperforming its spectrogram (0.8066) and waveform (0.8223) baselines. Moreover, it demonstrates superior robustness to domain shift on the challenging PASCAL dataset, maintaining an ROC-AUC of 0.7181 while the spectrogram baseline collapses (0.4873). Fusing complementary representations thus provides a strong inductive bias, enabling the creation of efficient, generalizable classifiers without requiring large-scale pre-training.

View on arXivView PDF

Code Implementations(1)

Saiful185/AudioFuseCommunity77%
MIT

AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Biomedical Audio Classification

00Jupyter NotebookSep 17, 20256 months agoMIT
audio-signal-processingbiomedical-signal-processingconvolutional-neural-networksfusion-modelsmultimodal-learning+4 more

Similar Papers

Comparative Analysis of CNN and Transformer Architectures with Heart Cycle Normalization for Automated Phonocardiogram Classification

83% match2025

Tuning In: Analysis of Audio Classifier Performance in Clinical Settings with Limited Data

83% match2024

Spectral and Rhythm Features for Audio Classification with Deep Convolutional Neural Networks

82% match2024

Spectral and Rhythm Feature Performance Evaluation for Category and Class Level Audio Classification with Deep Convolutional Neural Networks

81% match2025

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

80% match2 repos242023

Geometry-Aware Optimization for Respiratory Sound Classification: Enhancing Sensitivity with SAM-Optimized Audio Spectrogram Transformers

80% match2025

Cite this paper

@article{siddiqui2025audiofuse,
  title  = {AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Robust Phonocardiogram Classification},
  author = {Md. Saiful Bari Siddiqui and Utsab Saha},
  year   = {2025},
  eprint = {2509.23454},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.23454v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.23454.svg)](https://codeofpaper.com/paper/2509.23454)

Discussion