The paper investigates how multilingual large language models can unintentionally switch languages during generation. It compares three techniques—ValSel, FreqSel, and AnnSel—for pinpointing latent variables that control language choice in cross‑layer transcoders. Using new multilingual benchmarks and targeted interventions on Gemma‑2‑2B and Qwen3‑4B, the study finds all methods can steer output language, with FreqSel performing best and AnnSel providing interpretable selections via explicit annotations.
By Ryo Mitsuhashi, Sabri Boughorbel, Majd Hawasly
arXiv:2601.16390v2 Announce Type: replace-cross
Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...
By Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar
arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.
By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique
The paper investigates whether training small decoder-only transformers on code‑switched text can induce cross‑lingual alignment. Using two 100‑million‑word multilingual corpora—one a mix of English, Dutch, and Chinese BabyBabelLM data, and another generated by inserting word‑ and sentence‑level code‑switching via an LLM—the authors find that code‑switched training aligns representations of parallel text, especially across different scripts, and that this alignment persists when later training on monolingual documents. A curriculum that progresses from word‑level code‑switching to sentence‑level code‑switching and finally to monolingual data yields models that outperform baselines on the BabyLM evaluation suite, demonstrating that code‑switching curriculum learning is an effective data augmentation strategy for multilingual pretraining.
By Dries Rooryck, Alex Cai, Yonatan Belinkov, David Alvarez-Melis, Kiant\'e Brantley
arXiv:2510. 05678v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have achieved notable progress in multilingual settings, their performance remains uneven across languages as LLMs often rely on English-centric latent representations.
By Haneul Yoo, Jiho Jin, Kyunghyun Cho, Alice Oh
The paper addresses the challenge of identifying code‑switched utterances in language identification systems. It revisits the MaskLID approach, highlighting its overreliance on word‑level language association scores, and reformulates its optimization as an Integer Linear Program to incorporate clear, interpretable constraints. These enhancements significantly improve performance across ten diverse languages on code‑switching benchmarks, with the authors releasing code and data for reproducibility.
By Joanna Rado{\l}a, Josep Maria Crego, Fran\c{c}ois Yvon