Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features

Steffen Freisinger, Philipp Seeberger, Tobias Bocklet, Korbinian Riedhammer
2/6/2026
cs.CLeess.AS

No official code found. Best implementation: steffrs/multimodal-topic-segmentation (0 stars, last commit 2026-04-28).

Abstract

Spoken content, such as online videos and podcasts, often spans multiple topics, which makes automatic topic segmentation essential for user navigation and downstream applications. However, current methods do not fully leverage acoustic features, leaving room for improvement. We propose a multi-modal approach that fine-tunes both a text encoder and a Siamese audio encoder, capturing acoustic cues around sentence boundaries. Experiments on a large-scale dataset of YouTube videos show substantial gains over text-only and multi-modal baselines. Our model also proves more resilient to ASR noise and outperforms a larger text-only baseline on three additional datasets in Portuguese, German, and English, underscoring the value of learned acoustic features for robust topic segmentation.

View on arXivView PDF

Code Implementations(1)

steffrs/multimodal-topic-segmentationCommunity68%

Repository for the paper "Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features"

00PythonJan 28, 20265 months ago

Similar Papers

Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation

81% match2023

Beyond Transcripts: A Renewed Perspective on Audio Chaptering

80% match2026

Advancing Topic Segmentation of Broadcasted Speech with Multilingual Semantic Embeddings

80% match1 repo32024

Chunking Strategies for Multimodal AI Systems

80% match2025

Multimodal Fusion and Coherence Modeling for Video Topic Segmentation

79% match2024

Topic Segmentation Model Focusing on Local Context

79% match2023

Cite this paper

@article{freisinger2026reading,
  title  = {Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features},
  author = {Steffen Freisinger and Philipp Seeberger and Tobias Bocklet and Korbinian Riedhammer},
  year   = {2026},
  eprint = {2602.06647},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2602.06647v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2602.06647.svg)](https://codeofpaper.com/paper/2602.06647)

Discussion