The paper proposes the interlingua hypothesis, suggesting that large language models translate by encoding a source sentence into a latent, task‑agnostic feature space and then decoding a target sentence from that space. Three lines of evidence support this: (1) BLEU variance across language pairs is largely explained by language‑specific competences without pair‑specific interactions; (2) many model components influence both monolingual and translation tasks; and (3) fine‑tuning on monolingual data recovers most translation gains seen with aligned documents. These findings converge to support the hypothesis and point toward new ways to understand and improve LLM translation.
By Jacob Brinton, Jannik Brinkmann, Mark Crovella, Aaron Mueller
arXiv:2607. 19243v1 Announce Type: cross Abstract: Although Large Language Models (LLMs) demonstrate remarkable multilingual fluency, their internal knowledge representations remain disproportionately biased toward high-resource languages.
By Alexander Manev
The paper introduces Centroid Intervention Fusion (CIF), a framework that merges multiple multilingual intervention projections into a single language-shared operator for inference-time modification of large language models. CIF improves cross-lingual transfer without updating model parameters and achieves up to +3.378 percentage points better performance than prior pairwise intervention baselines across several benchmarks, including low-resource languages. The authors provide code at https://github.com/VRCMF/CIF.git.
By Wei Sun, Marie-Francine Moens
The paper presents a unified evaluation of cross‑lingual consistency (CLC) enhancement methods for multilingual language models, covering inference‑time interventions and post‑training approaches across three model families and three closed‑form benchmarks. Results indicate that post‑training methods, especially direct distribution alignment, consistently improve CLC across all model‑dataset combinations, while other methods are more sensitive to answer format and language coverage. The study also examines the impact of CLC enhancement on culturally diverse question answering, finding no systematic degradation in controlled settings but occasional accuracy drops in open‑ended generation, particularly for non‑English responses.
By Jirui Qi, Mingyang Wang, Hinrich Sch\"utze, Raquel Fern\'andez, Arianna Bisazza
The paper investigates multilingual confidence calibration in large language models, revealing that non‑English languages are systematically less well calibrated than English. By analyzing internal representations, the authors find that late‑intermediate layers provide a more reliable confidence signal than the final layer, which is biased by English‑centric training. They propose training‑free methods such as Language‑Aware Confidence Ensemble (LACE) to adaptively select optimal layers per language, aiming to improve global equity and trustworthiness of LLMs.
By Ej Zhou, Caiqi Zhang, Tiancheng Hu, Chengzu Li, Nigel Collier, Ivan Vuli\'c, Anna Korhonen
arXiv:2510. 05678v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have achieved notable progress in multilingual settings, their performance remains uneven across languages as LLMs often rely on English-centric latent representations.
By Haneul Yoo, Jiho Jin, Kyunghyun Cho, Alice Oh
SWORD is a new benchmark that tests large language models’ ability to reject factually incorrect statements across eight major languages by distorting Wikidata triples. The benchmark reveals that models often perform better on semantically plausible distortions than on random ones, indicating a reliance on distributional familiarity rather than true factual verification. It also shows significant performance drops for East Asian languages, with gaps up to 28 percentage points, highlighting asymmetric multilingual factual reasoning capabilities.
By Sanghyeok Park, Minji Kang, Hosung Kwak, Jinhyuk Yun
The study investigates whether monolingual language models, trained without joint multilingual objectives, develop cross-lingual alignment. By evaluating models such as Goldfish and independently built monolingual systems, the authors find that alignable representational geometry emerges across layers, strengthening with larger data, larger models, or closer linguistic proximity. A single Procrustes rotation on parallel sentences can map hidden states between models, and applying this rotation to a German model’s residuals swaps factual predictions to those of the donor English model, demonstrating functional transfer.
By Ej Zhou, Suchir Salhan, Catherine Arnett, Anna Korhonen
arXiv:2608.03446v2 Announce Type: replace
Abstract: Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the giv...
By Adnan Al Ali, Kathy H\"ammerl, Jind\v{r}ich Libovick\'y, Alexander Fraser
The study compares an English-only and a bilingual decoder-only model, each 310 M parameters, trained on eight diverse languages while controlling for English exposure, compute, and document overlap. After aligning on shared English vocabulary, the authors find that token embeddings appear similar, but the deeper hidden states used for prediction diverge across models. This hidden‑state mismatch grows through middle transformer layers and persists despite controls, indicating that contextual processing differs between the models.
"whyItMatters":"The findings show that embedding alignment can conceal significant internal representation differences, which is crucial for any downstream work that assumes aligned multilingual models are interchangeable."
By Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos
The paper investigates how multilingual large language models can be guided to reason more reliably in low- to mid-resource languages by selecting appropriate language modes during inference. Experiments with LLaMA and Qwen models show that using English context can correct errors from non‑English comprehension, but adding redundant bilingual context can cause interference. To balance this trade‑off, the authors propose Reliability‑Aware Adaptive Inference (RAAI), a training‑free test‑time framework that routes prompts based on Expected Calibration Error and gates reasoning with a mid‑layer Risk Index, achieving up to 37.7% accuracy gains and reduced calibration error on low‑resource languages.
By Ekata Mitra, Ameeta Agrawal
The study investigates whether multilingual language models solve mathematical problems using shared or language‑specific internal representations. By replaying reasoning traces across six languages and applying Centered Kernel Alignment, the authors identify layers with cross‑language alignment and train two sparse autoencoders— a baseline and a new Geometry‑Invariant SAE (GI‑SAE) that adds an InfoNCE loss to encourage language‑agnostic activations. Functional interchangeability tests reveal that higher geometric similarity does not always translate to interchangeable features, and cross‑language sharing varies by model and depth, with GI‑SAE amplifying existing structure in some models but not others.
By Igor Bogdanov, Changcheng Huang