Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India

Kaushal Bhogale, Manas Dhir, Amritansh Walecha, Manmeet Kaur, Vanshika Chhabra +8 more
4/21/2026
cs.CLcs.SDeess.AS

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Existing Indic ASR benchmarks often use scripted, clean speech and leaderboard driven evaluation that encourages dataset specific overfitting. In addition, strict single reference WER penalizes natural spelling variation in Indian languages, including non standardized spellings of code-mixed English origin words. To address these limitations, we introduce Voice of India, a closed source benchmark built from unscripted telephonic conversations covering 15 major Indian languages across 139 regional clusters. The dataset contains 306230 utterances, totaling 536 hours of speech from 36691 speakers with transcripts accounting for spelling variations. We also analyze performance geographically at the district level, revealing disparities. Finally, we provide detailed analysis across factors such as audio quality, speaking rate, gender, and device type, highlighting where current ASR systems struggle and offering insights for improving real world Indic ASR systems.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition

87% match2026

Benchmarking Automatic Speech Recognition Tools for Iberian Languages

85% match2026

Investigating the Sensitivity of Automatic Speech Recognition Systems to Phonetic Variation in L2 Englishes

85% match2023

A Deep Dive into the Disparity of Word Error Rates Across Thousands of NPTEL MOOC Videos

85% match1 repo12023

DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Units

85% match2026

Evaluating Bias in Phoneme-Based Automatic Speech Recognition Systems: An Analysis of IPA Transcription Models

85% match2026

Cite this paper

@article{bhogale2026voice,
  title  = {Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India},
  author = {Kaushal Bhogale and Manas Dhir and Amritansh Walecha and Manmeet Kaur and Vanshika Chhabra and Aaditya Pareek and Hanuman Sidh and Sagar Jain and Bhaskar Singh and Utkarsh Singh and Tahir Javed and Shobhit Banga and Mitesh M. Khapra},
  year   = {2026},
  eprint = {2604.19151},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2604.19151v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2604.19151.svg)](https://codeofpaper.com/paper/2604.19151)

Discussion