Music- and Lyrics-driven Dance Synthesis

Wenjie Yin, Qingyuan Yao, Yi Yu, Hang Yin, Danica Kragic +1 more
9/30/2023
cs.MMcs.GRcs.LG

Official code: yy1lab/LMD (4 stars, Apache-2.0, last commit 2023-09-28). 1 more implementation below.

Abstract

Lyrics often convey information about the songs that are beyond the auditory dimension, enriching the semantic meaning of movements and musical themes. Such insights are important in the dance choreography domain. However, most existing dance synthesis methods mainly focus on music-to-dance generation, without considering the semantic information. To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics. To the best of our knowledge, this is the first dataset with triplet information including dance motion, music, and lyrics. Additionally, we showcase a cross-modal diffusion-based network designed to generate 3D dance motion conditioned on music and lyrics. The proposed JustLMD dataset encompasses 4.6 hours of 3D dance motion in 1867 sequences, accompanied by musical tracks and their corresponding English lyrics.

View on arXivView PDF

Code Implementations(2)

yy1lab/LMDOfficial
Apache-2.0

Music- and Lyrics-driven Dance Synthesis

40PythonSep 27, 20233 years agoApache-2.0
YIN95/LMDCommunity63%
Apache-2.0

Music- and Lyrics-driven Dance Synthesis

31PythonSep 27, 20233 years agoApache-2.0

Similar Papers

LM2D: Lyrics- and Music-Driven Dance Synthesis

90% match2024

TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration

84% match1 repo1092023

DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance

84% match1 repo1172024

MAGMA: Music Aligned Generative Motion Autodecoder

83% match2023

Listen to Rhythm, Choose Movements: Autoregressive Multimodal Dance Generation via Diffusion and Mamba with Decoupled Dance Dataset

83% match2026

Bidirectional Autoregressive Diffusion Model for Dance Generation

83% match2024

Cite this paper

@article{yin2023music,
  title  = {Music- and Lyrics-driven Dance Synthesis},
  author = {Wenjie Yin and Qingyuan Yao and Yi Yu and Hang Yin and Danica Kragic and Mårten Björkman},
  year   = {2023},
  eprint = {2310.00455},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.00455v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.00455.svg)](https://codeofpaper.com/paper/2310.00455)

Discussion