Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics

Tibor Krols, Yana Nikolova, Ninell Oldenburg
2/26/2023
cs.SDcs.CLcs.MM

No official code found. Best implementation: Tibor-Krols/CogSci2-Spotify (13 stars, last commit 2023-02-19).

Abstract

This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify API, combined lyrics features including sentiment, TF-IDF, and Anew to predict valence and arousal (Russell, 1980) scores on the Deezer Mood Detection Dataset (DMDD) (Delbouys et al., 2018) with 4 different regression models. We find that out of the 11 high-level song features, mainly 5 contribute to the performance, multi-modal features do better than audio alone when predicting valence. We made our code publically available.

View on arXivView PDF

Code Implementations(1)

Tibor-Krols/CogSci2-SpotifyCommunity100%

Sentiment analysis of song lyrics compared to auditory track features and valence

131Python, Jupyter NotebookApr 26, 20223 years ago

Similar Papers

Tollywood Emotions: Annotation of Valence-Arousal in Telugu Song Lyrics

90% match2023

Exploring the Emotional Landscape of Music: An Analysis of Valence Trends and Genre Variations in Spotify Music Data

87% match2023

Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition

86% match1 repo322024

Joint Learning of Emotions in Music and Generalized Sounds

86% match1 repo22024

Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better

85% match2024

Feature-Based Dual Visual Feature Extraction Model for Compound Multimodal Emotion Recognition

85% match2025

Cite this paper

@article{krols2023multimodality,
  title  = {Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics},
  author = {Tibor Krols and Yana Nikolova and Ninell Oldenburg},
  year   = {2023},
  eprint = {2302.13321},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.13321v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.13321.svg)](https://codeofpaper.com/paper/2302.13321)

Discussion