End to end Hindi to English speech conversion using Bark, mBART and a finetuned XLSR Wav2Vec2

Aniket Tathe, Anand Kamble, Suyash Kumbharkar, Atharva Bhandare, Anirban C. Mitra
1/11/2024
eess.AScs.AIcs.CL

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Speech has long been a barrier to effective communication and connection, persisting as a challenge in our increasingly interconnected world. This research paper introduces a transformative solution to this persistent obstacle an end-to-end speech conversion framework tailored for Hindi-to-English translation, culminating in the synthesis of English audio. By integrating cutting-edge technologies such as XLSR Wav2Vec2 for automatic speech recognition (ASR), mBART for neural machine translation (NMT), and a Text-to-Speech (TTS) synthesis component, this framework offers a unified and seamless approach to cross-lingual communication. We delve into the intricate details of each component, elucidating their individual contributions and exploring the synergies that enable a fluid transition from spoken Hindi to synthesized English audio.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving

83% match1 repo22024

HITSZ's End-To-End Speech Translation Systems Combining Sequence-to-Sequence Auto Speech Recognition Model and Indic Large Language Model for IWSLT 2025 in Indic Track

83% match2025

Transcription and translation of videos using fine-tuned XLSR Wav2Vec2 on custom dataset and mBART

81% match2024

MahaTTS: A Unified Framework for Multilingual Text-to-Speech Synthesis

81% match1 repo32025

Code-Mixed Text to Speech Synthesis under Low-Resource Constraints

80% match2023

Direct Punjabi to English speech translation using discrete units

80% match2024

Cite this paper

@article{tathe2024hindi,
  title  = {End to end Hindi to English speech conversion using Bark, mBART and a finetuned XLSR Wav2Vec2},
  author = {Aniket Tathe and Anand Kamble and Suyash Kumbharkar and Atharva Bhandare and Anirban C. Mitra},
  year   = {2024},
  eprint = {2401.06183},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.06183v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.06183.svg)](https://codeofpaper.com/paper/2401.06183)

Discussion