arXiv:2604.15490v2 Announce Type: replace
Abstract: Recent developments in reasoning capabilities have enabled large language models to solve increasingly complex mathematical, symbolic, and logical...
By Eleanor M. Lin, David Jurgens
The paper investigates whether training small decoder-only transformers on code‑switched text can induce cross‑lingual alignment. Using two 100‑million‑word multilingual corpora—one a mix of English, Dutch, and Chinese BabyBabelLM data, and another generated by inserting word‑ and sentence‑level code‑switching via an LLM—the authors find that code‑switched training aligns representations of parallel text, especially across different scripts, and that this alignment persists when later training on monolingual documents. A curriculum that progresses from word‑level code‑switching to sentence‑level code‑switching and finally to monolingual data yields models that outperform baselines on the BabyLM evaluation suite, demonstrating that code‑switching curriculum learning is an effective data augmentation strategy for multilingual pretraining.
By Dries Rooryck, Alex Cai, Yonatan Belinkov, David Alvarez-Melis, Kiant\'e Brantley
The study investigates the performance gap between native-language reasoning and English-pivoted reasoning in large language models. By creating extensive multilingual reasoning datasets and fine‑tuning specialists on Qwen/Qwen3-8B-Base, the authors find that the native reasoning gap is much smaller (1.9–3.5%) than previously reported. They analyze weight‑space changes, discover a language‑agnostic reasoning core in the middle layers, and propose a Layer Swap technique that transfers these mid‑layer updates from an English specialist to native specialists, effectively closing most of the gap while maintaining native chain‑of‑thought output.
By Maxence Lasbordes, Am\'elie Chatelain, Djam\'e Seddah
arXiv:2608. 00533v1 Announce Type: cross Abstract: Large Language Models have achieved substantial progress in reasoning capabilities.
By Sean Gip Lim, William Chandra Tjhi, Hai Leong Chieu
The paper investigates why large reasoning language models struggle to transfer parametric knowledge across different scripts. Through observational data and regression analysis on ECLeKTic and MultiLoKo datasets, the authors find that script mismatch—not language family—is the main predictor of transfer failure when controlling for model capability and question difficulty. By providing key entities in the source language and training models to reason about transliteration ambiguities, they demonstrate a reduction in the cross‑script transfer gap, suggesting that post‑training improvements can enhance cross‑lingual knowledge transfer.
By Lucas Bandarkar, Alan Ansell, Trevor Cohn
The paper introduces Centroid Intervention Fusion (CIF), a framework that merges multiple multilingual intervention projections into a single language-shared operator for inference-time modification of large language models. CIF improves cross-lingual transfer without updating model parameters and achieves up to +3.378 percentage points better performance than prior pairwise intervention baselines across several benchmarks, including low-resource languages. The authors provide code at https://github.com/VRCMF/CIF.git.
By Wei Sun, Marie-Francine Moens