MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning

Dichucheng Li, Yinghao Ma, Weixing Wei, Qiuqiang Kong, Yulun Wu +4 more
10/15/2023
cs.SDcs.AIcs.LG

No official code found. Best implementation: LiDCC/MERTech (11 stars, last commit 2024-06-14).

Abstract

Instrument playing techniques (IPTs) constitute a pivotal component of musical expression. However, the development of automatic IPT detection methods suffers from limited labeled data and inherent class imbalance issues. In this paper, we propose to apply a self-supervised learning model pre-trained on large-scale unlabeled music data and finetune it on IPT detection tasks. This approach addresses data scarcity and class imbalance challenges. Recognizing the significance of pitch in capturing the nuances of IPTs and the importance of onset in locating IPT events, we investigate multi-task finetuning with pitch and onset detection as auxiliary tasks. Additionally, we apply a post-processing approach for event-level prediction, where an IPT activation initiates an event only if the onset output confirms an onset in that frame. Our method outperforms prior approaches in both frame-level and event-level metrics across multiple IPT benchmark datasets. Further experiments demonstrate the efficacy of multi-task finetuning on each IPT class.

View on arXivView PDF

Code Implementations(1)

LiDCC/MERTechCommunity60%

Official code of ICASSP 2024 paper "MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning"

114PythonSep 4, 20232 years ago

Similar Papers

Toward Fully Self-Supervised Multi-Pitch Estimation

82% match1 repo252024

Investigating an Overfitting and Degeneration Phenomenon in Self-Supervised Multi-Pitch Estimation

81% match2025

Detecting Music Performance Errors with Transformers

80% match1 repo232025

A Hierarchical Deep Learning Approach for Minority Instrument Detection

79% match1 repo22025

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

79% match1 repo5102023

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

79% match2026

Cite this paper

@article{li2023mertech,
  title  = {MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning},
  author = {Dichucheng Li and Yinghao Ma and Weixing Wei and Qiuqiang Kong and Yulun Wu and Mingjin Che and Fan Xia and Emmanouil Benetos and Wei Li},
  year   = {2023},
  eprint = {2310.09853},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.09853v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.09853.svg)](https://codeofpaper.com/paper/2310.09853)

Discussion