arXiv Computation and Language

How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?

The paper investigates how mathematical reasoning improvements from Reinforcement Post-Training (RPT) in English-centric large language models (LLMs) transfer to other languages. Experiments show that transfer varies greatly depending on the initial model, target language, and training approach, and that stronger English initialization does not guarantee better transfer efficiency. Adding a single parallel language to the training data yields the largest boost, with additional languages providing smaller but consistent gains.

arXiv Computation and Language

Aug 27

Rethinking the Multilingual Reasoning Gap with Layer Swap

The study investigates the performance gap between native-language reasoning and English-pivoted reasoning in large language models. By creating extensive multilingual reasoning datasets and fine‑tuning specialists on Qwen/Qwen3-8B-Base, the authors find that the native reasoning gap is much smaller (1.9–3.5%) than previously reported. They analyze weight‑space changes, discover a language‑agnostic reasoning core in the middle layers, and propose a Layer Swap technique that transfers these mid‑layer updates from an English specialist to native specialists, effectively closing most of the gap while maintaining native chain‑of‑thought output.

By Maxence Lasbordes, Am\'elie Chatelain, Djam\'e Seddah

llms fine-tuning

arXiv AI

Sep 30

What Does Post-Training Change in Multilingual Reasoning?

arXiv:2609.37104v1 Announce Type: cross Abstract: Open-source reasoning models provide unequal access to reasoning capability across languages. When a model can solve a problem but cannot deliver a c...

By Hongyang Li, Xiao Li, Caesar Wu, Gr\'egoire Danoy, Pascal Bouvry

reinforcement-learning fine-tuning

arXiv Machine Learning

Aug 17

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings

arXiv:2608. 13698v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capabilities of pretrained language models but current studies remain heavily English-centric.

By Konstantin Dobler, Federico Scozzafava, Jonathan Janke, Mohamed Ali, Simon Lehnerer

llms reinforcement-learning

arXiv AI

Jun 16

AdaMame: A Training Recipe for Adaptive Multilingual Reasoning

arXiv:2606. 15080v1 Announce Type: cross Abstract: While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse.

By Dayeon Ki, Kevin Duh, Marine Carpuat

reinforcement-learning fine-tuning benchmarks safety

Hugging Face Trending Papers

Jun 1

Learning When to Translate for Multilingual Reasoning

Reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, but still exhibit substantial multilingual reasoning gaps, largely due to language-understanding failures in non-English inputs. English translation can mitigate these failures by expressing non-English inputs in a form that RLMs can more reliably interpret, yet translating every input is unnecessary when the model can reason reliably from the original query.

llms reinforcement-learning benchmarks

arXiv Computation and Language

Sep 22

What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features

arXiv:2604.04720v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps ca...

By Dayeon Ki, Kevin Duh, Marine Carpuat

benchmarks safety

arXiv AI

Jul 8

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.

By Daryna Dementieva, Nikolay Babakov, Kathy H\"ammerl, Ilseyar Alimova, Jind\v{r}ich Libovick\'y, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan \"Ozer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser

llms benchmarks

arXiv AI

Jun 2

Learning When to Translate for Multilingual Reasoning

arXiv:2606. 02465v1 Announce Type: cross Abstract: Reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, but still exhibit substantial multilingual reasoning gaps, largely due to language-understanding failures in non-English inputs.

By Deokhyung Kang, Hyounghun Kim, Gary Geunbae Lee

llms reinforcement-learning benchmarks

arXiv Computation and Language

Sep 22

English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training

arXiv:2604.13286v2 Announce Type: replace Abstract: Despite the widespread multilingual deployment of large language models, post-training pipelines remain predominantly English-centric, contributing...

By Mehak Dhaliwal, Shashwat Chaurasia, Yao Qin, Dezhi Hong, Thomas Butler

llms fine-tuning

arXiv Machine Learning

Aug 7

On-Policy Delta Distillation for Multilingual Math Reasoning

arXiv:2608. 05802v1 Announce Type: cross Abstract: On-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexplored.

By Byeongho Heo, Jaehui Hwang, Sangdoo Yun, Dongyoon Han

llms reinforcement-learning efficiency

arXiv Computation and Language

Sep 10

Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning

arXiv:2609.10445v1 Announce Type: new Abstract: Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: m...

By Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Berard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer

llms fine-tuning benchmarks

arXiv AI

Jul 8

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

arXiv:2605. 19723v2 Announce Type: replace-cross Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems.

By Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima

llms fine-tuning benchmarks

Related stories