Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes

Kevin Glocker, Aaricia Herygers, Munir Georges
6/7/2023
cs.CLcs.SDeess.AS

No official code found. Best implementation: kgnlp/allophant (30 stars, MIT, last commit 2025-03-14).

Abstract

This paper proposes Allophant, a multilingual phoneme recognizer. It requires only a phoneme inventory for cross-lingual transfer to a target language, allowing for low-resource recognition. The architecture combines a compositional phone embedding approach with individually supervised phonetic attribute classifiers in a multi-task architecture. We also introduce Allophoible, an extension of the PHOIBLE database. When combined with a distance based mapping approach for grapheme-to-phoneme outputs, it allows us to train on PHOIBLE inventories directly. By training and evaluating on 34 languages, we found that the addition of multi-task learning improves the model's capability of being applied to unseen phonemes and phoneme inventories. On supervised languages we achieve phoneme error rate improvements of 11 percentage points (pp.) compared to a baseline without multi-task learning. Evaluation of zero-shot transfer on 84 languages yielded a decrease in PER of 2.63 pp. over the baseline.

View on arXivView PDF

Code Implementations(1)

kgnlp/allophantCommunity100%
MIT

A multilingual phoneme recognizer capable of generalizing zero-shot to unseen phoneme inventories.

304Rust, PythonMay 29, 20231 years agoMIT
cross-lingualmachine-learningmultilingualneural-networksphoneme-recognition+2 more

Similar Papers

Advancing LLM-based phoneme-to-grapheme for multilingual speech recognition

88% match2026

Optimizing Two-Pass Cross-Lingual Transfer Learning: Phoneme Recognition and Phoneme to Grapheme Translation

88% match2023

Learning Cross-lingual Mappings for Data Augmentation to Improve Low-Resource Speech Recognition

87% match2023

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark

86% match2 repos3112023

The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech

86% match2023

Visual Speech Recognition for Languages with Limited Labeled Data using Automatic Labels from Whisper

86% match1 repo172023

Cite this paper

@article{glocker2023allophant,
  title  = {Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes},
  author = {Kevin Glocker and Aaricia Herygers and Munir Georges},
  year   = {2023},
  eprint = {2306.04306},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.04306v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.04306.svg)](https://codeofpaper.com/paper/2306.04306)

Discussion