The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation

Ilaria Manco, Benno Weck, SeungHeon Doh, Minz Won, Yixiao Zhang +8 more
11/16/2023
cs.SDcs.AIcs.CL

No official code found. Best implementation: mulab-mir/song-describer-dataset (175 stars, MIT, last commit 2023-12-22).

Abstract

We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural language descriptions of 706 music recordings, all publicly accessible and released under Creative Common licenses. To showcase the use of our dataset, we benchmark popular models on three key music-and-language tasks (music captioning, text-to-music generation and music-language retrieval). Our experiments highlight the importance of cross-dataset evaluation and offer insights into how researchers can use SDD to gain a broader understanding of model performance.

View on arXivView PDF

Code Implementations(1)

mulab-mir/song-describer-datasetCommunity100%
MIT

The Song Describer dataset is an evaluation dataset made of ~1.1k captions for 706 permissively licensed music recordings.

1754Shell, PythonOct 30, 20232 years agoMIT
audio-textevaluationmusic-dataset

Similar Papers

JamendoMaxCaps: A Large Scale Music-caption Dataset with Imputed Metadata

89% match1 repo532025

SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval

89% match2026

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

87% match1 repo2622023

MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response

86% match1 repo502023

Developing a Multilingual Dataset and Evaluation Metrics for Code-Switching: A Focus on Hong Kong's Polylingual Dynamics

85% match1 repo412023

Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning

85% match2023

Cite this paper

@article{manco2023song,
  title  = {The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation},
  author = {Ilaria Manco and Benno Weck and SeungHeon Doh and Minz Won and Yixiao Zhang and Dmitry Bogdanov and Yusong Wu and Ke Chen and Philip Tovstogan and Emmanouil Benetos and Elio Quinton and György Fazekas and Juhan Nam},
  year   = {2023},
  eprint = {2311.10057},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.10057v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.10057.svg)](https://codeofpaper.com/paper/2311.10057)

Discussion