Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0

Marianne de Heer Kloots, Willem Zuidema
7/3/2024
cs.CLcs.AIcs.SD

No official code found. Best implementation: mdhk/phonotactic-sensitivity (7 stars, last commit 2024-09-04).

Abstract

What do deep neural speech models know about phonology? Existing work has examined the encoding of individual linguistic units such as phonemes in these models. Here we investigate interactions between units. Inspired by classic experiments on human speech perception, we study how Wav2Vec2 resolves phonotactic constraints. We synthesize sounds on an acoustic continuum between /l/ and /r/ and embed them in controlled contexts where only /l/, only /r/, or neither occur in English. Like humans, Wav2Vec2 models show a bias towards the phonotactically admissable category in processing such ambiguous sounds. Using simple measures to analyze model internals on the level of individual stimuli, we find that this bias emerges in early layers of the model's Transformer module. This effect is amplified by ASR finetuning but also present in fully self-supervised models. Our approach demonstrates how controlled stimulus designs can help localize specific linguistic knowledge in neural speech models.

View on arXivView PDF

Code Implementations(1)

mdhk/phonotactic-sensitivityCommunity100%

Code and materials for our Interspeech 2024 paper on 'Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0'

71PythonJun 13, 20242 years ago

Similar Papers

Perception of Phonological Assimilation by Neural Speech Recognition Models

86% match2024

Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representations

81% match2025

Distinct social-linguistic processing between humans and large audio-language models: Evidence from model-brain alignment

81% match2025

Self-supervised Neural Factor Analysis for Disentangling Utterance-level Speech Representations

81% match2023

Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models

81% match2025

Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition

81% match2023

Cite this paper

@article{kloots2024humanlike,
  title  = {Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0},
  author = {Marianne de Heer Kloots and Willem Zuidema},
  year   = {2024},
  eprint = {2407.03005},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.03005v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.03005.svg)](https://codeofpaper.com/paper/2407.03005)

Discussion