← Back to all news
arXiv Machine Learning By Moran Yanuka, Raja Giryes, Moris Alper

Phonological Interference in Multilingual Speech Models

Read the original on arXiv Machine Learning →

The paper investigates a failure mode called phonological interference in phoneme‑level multilingual speech models, where models incorrectly impose a single language’s phonology on mixed or unseen language input. Experiments show significant loss of language‑specific phonemes in code‑switched and unseen‑language scenarios, and the authors trace this effect to a low‑dimensional subspace of internal activations. They propose a windowed language estimation (WLE) repair that reduces interference by 34–69% while preserving monolingual performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 14

An Empirical Recipe for Universal Phone Recognition

arXiv:2603. 29042v2 Announce Type: replace-cross Abstract: Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks, yet robust performance remains elusive.

By Shikhar Bharadwaj, Chin-Jou Li, Kwanghee Choi, Eunjung Yeo, William Chen, Shinji Watanabe, David R. Mortensen
benchmarks
More like this →
arXiv Machine Learning
6d ago

Phonologically Informed Tokenization for German Speech Recognition: A Cross-Domain Study

The study evaluates phonologically informed tokenization for German speech recognition using a wav2vec 2.0 backbone fine‑tuned with CTC. Three tokenizer families—pretrained multilingual characters, data‑driven BPE, and phonologically informed units from syllabification and grapheme‑to‑phoneme conversion—are compared across 40 fine‑tunes on three German test sets (in‑domain read, dialectal spontaneous, and standard spontaneous speech). Results show that phonologically informed tokenizers match BPE and character baselines in‑domain, but under domain shift their performance depends on vocabulary size rather than linguistic variation, with syllable‑aware tokenization benefiting dialectal and spontaneous speech at small vocabularies; a phoneme‑level confusion analysis indicates that the acoustic encoder dominates error patterns.

By Christopher Witzl, Tobias Bocklet, Korbinian Riedhammer
nlpfine-tuningmultimodal
More like this →
arXiv Computation and Language
Sep 25

Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition

The paper introduces task-informed parameter-efficient fine-tuning methods for low-resource speech recognition by applying Fisher-Whitened Cross-Covariance Analysis (FCCA) to Whisper and Qwen3-ASR. Two extensions—Asymmetric-Coupled FCCA (AC‑FCCA) and Adaptive‑Rank FCCA (AR‑FCCA)—are proposed to exploit cross‑layer sharing and adapt rank allocation within a fixed parameter budget. Experiments on multilingual datasets show that standard FCCA matches or surpasses LoRA, while AR‑FCCA consistently improves performance across models without increasing trainable parameters.

By Asmee Mishra, Mengjie Qian, Brechtje Post, Kate Knill
fine-tuningmultimodal
More like this →