RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonations

Ashwin Sankar, Yoach Lacombe, Sherry Thomas, Praveen Srinivasa Varadhan, Sanchit Gandhi +1 more
5/24/2025
cs.CL

No confident code implementation found for this paper.

Abstract

We introduce RASMALAI, a large-scale speech dataset with rich text descriptions, designed to advance controllable and expressive text-to-speech (TTS) synthesis for 23 Indian languages and English. It comprises 13,000 hours of speech and 24 million text-description annotations with fine-grained attributes like speaker identity, accent, emotion, style, and background conditions. Using RASMALAI, we develop IndicParlerTTS, the first open-source, text-description-guided TTS for Indian languages. Systematic evaluation demonstrates its ability to generate high-quality speech for named speakers, reliably follow text descriptions and accurately synthesize specified attributes. Additionally, it effectively transfers expressive characteristics both within and across languages. IndicParlerTTS consistently achieves strong performance across these evaluations, setting a new standard for controllable multilingual expressive speech synthesis in Indian languages.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings

86% match1 repo152024

IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS

85% match1 repo652024

Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages

85% match1 repo302024

MahaTTS: A Unified Framework for Multilingual Text-to-Speech Synthesis

84% match1 repo32025

IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages

83% match2024

IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages

82% match1 repo4132024

Cite this paper

@article{sankar2025rasmalai,
  title  = {RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonations},
  author = {Ashwin Sankar and Yoach Lacombe and Sherry Thomas and Praveen Srinivasa Varadhan and Sanchit Gandhi and Mitesh M Khapra},
  year   = {2025},
  eprint = {2505.18609},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2505.18609v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2505.18609.svg)](https://codeofpaper.com/paper/2505.18609)

Discussion