Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings

Praveen Srinivasa Varadhan, Ashwin Sankar, Giri Raju, Mitesh M. Khapra
7/19/2024
cs.CLcs.LGcs.SD

No official code found. Best implementation: AI4Bharat/Rasa (15 stars, last commit 2025-03-06).

Abstract

We release Rasa, the first multilingual expressive TTS dataset for any Indian language, which contains 10 hours of neutral speech and 1-3 hours of expressive speech for each of the 6 Ekman emotions covering 3 languages: Assamese, Bengali, & Tamil. Our ablation studies reveal that just 1 hour of neutral and 30 minutes of expressive data can yield a Fair system as indicated by MUSHRA scores. Increasing neutral data to 10 hours, with minimal expressive data, significantly enhances expressiveness. This offers a practical recipe for resource-constrained languages, prioritizing easily obtainable neutral data alongside smaller amounts of expressive data. We show the importance of syllabically balanced data and pooling emotions to enhance expressiveness. We also highlight challenges in generating specific emotions, e.g., fear and surprise.

View on arXivView PDF

Code Implementations(1)

AI4Bharat/RasaCommunity100%

Expressive TTS Dataset for Assamese, Bengali, and Tamil.

156HTML, ShellJun 11, 20241 years ago

Similar Papers

RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonations

86% match2025

IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages

83% match2024

IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS

83% match1 repo652024

Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages

82% match1 repo302024

A2TTS: TTS for Low Resource Indian Languages

82% match2025

ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams

81% match1 repo52024

Cite this paper

@article{varadhan2024rasa,
  title  = {Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings},
  author = {Praveen Srinivasa Varadhan and Ashwin Sankar and Giri Raju and Mitesh M. Khapra},
  year   = {2024},
  eprint = {2407.14056},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.14056v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.14056.svg)](https://codeofpaper.com/paper/2407.14056)

Discussion