arXiv AI

Evaluating the Effect of Linguistic Relatedness on Cross-Lingual Transfer in Large Multilingual Automatic Speech Recognition

arXiv:2607. 04814v1 Announce Type: cross Abstract: Extending automatic speech recognition (ASR) to low-resource African languages is constrained by the prohibitive demands of data collection at scale.

arXiv Computation and Language
Aug 28

AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition

AfriSwitch is a 61.36‑hour, human‑transcribed benchmark of in‑the‑wild code‑switched speech covering 16 African languages and varieties, annotated with switch‑level English span tags, per‑utterance Code‑Mixing Index (CMI), and switch‑point counts. The corpus reveals that code‑switching behaviour varies widely across languages, with no single metric fully capturing how code‑switched a language is. Benchmarking five open and commercial multilingual ASR systems in a zero‑shot setting shows high word error rates, with the best system averaging 35.93% WER and none dropping below 24% on any language, indicating that Africa‑targeted training rather than model scale or nominal language coverage best predicts performance.

By Gabrial Zencha Ashungafac, Busayo Awobade, Tobi Olatunji
arXiv Computation and Language
Aug 27

One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography

The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.

By Muge Zhang, Aaron Jencks, Krishna Badikela, Yulia Tsvetkov, Sachin Kumar
arXiv Computation and Language
Aug 27

Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

The paper investigates how to fairly compare language models across languages, noting that current evaluation methods vary widely and lack empirical validation. By training controlled monolingual models on parallel data and testing multilingual LLMs, the authors find that many normalized metrics suffer from biases due to tokenization, encoding, and orthographic differences. Instead, they recommend using sentence‑level negative log‑likelihood over semantically equivalent sequences for more reliable cross‑lingual comparisons.

By Xiulin Yang, Ethan Gotlieb Wilcox, Catherine Arnett
arXiv AI
Aug 19

Language Family Matters: Evaluating LLM-Based ASR Across Linguistic Boundaries

The paper introduces a new strategy for connecting large language models (LLMs) to speech encoders in automatic speech recognition (ASR) systems by sharing a single connector across languages within the same linguistic family. This approach reduces the number of parameters needed compared to training a separate connector for each language, while improving generalization across different domains and real‑world corpora. Experiments with two multilingual LLMs and two speech datasets demonstrate that family‑based connectors are both efficient and effective for multilingual ASR deployment.

By Yuchen Zhang, Ravi Shekhar, Haralambos Mouratidis
arXiv AI
Aug 28

Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics

The paper investigates why large language models sometimes hallucinate when asked about facts in a language different from the one in which the facts were learned. By training small Transformer models on synthetic multilingual datasets, the authors show that the degree of correlation between facts and their learning language (informativeness) and the ease of language identification (extractability) determine whether models develop unified or separate representations across languages. Unified representations enable cross‑lingual fact transfer, while separate representations do not. The study proposes a unifying perspective on cross‑lingual transfer and suggests training methods to promote representational unification.

By Carter Blum, Katja Filippova, Ann Yuan, Asma Ghandeharioun, Julian Zimmert, Fred Zhang, Jessica Hoffmann, Tal Linzen, Martin Wattenberg, Lucas Dixon, Mor Geva