arXiv AI
Benchmarking Arabic Dialects for Math Problems
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
arXiv:2601.21225v4 Announce Type: replace-cross Abstract: Large language models have made substantial progress in mathematical reasoning. However, benchmark development for multilingual evaluation ha...
arXiv AI
Jul 8
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.
arXiv Computation and Language
Aug 27
Rethinking the Multilingual Reasoning Gap with Layer Swap
The study investigates the performance gap between native-language reasoning and English-pivoted reasoning in large language models. By creating extensive multilingual reasoning datasets and fine‑tuning specialists on Qwen/Qwen3-8B-Base, the authors find that the native reasoning gap is much smaller (1.9–3.5%) than previously reported. They analyze weight‑space changes, discover a language‑agnostic reasoning core in the middle layers, and propose a Layer Swap technique that transfers these mid‑layer updates from an English specialist to native specialists, effectively closing most of the gap while maintaining native chain‑of‑thought output.
arXiv Computation and Language
6d ago
How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?
The paper investigates how mathematical reasoning improvements from Reinforcement Post-Training (RPT) in English-centric large language models (LLMs) transfer to other languages. Experiments show that transfer varies greatly depending on the initial model, target language, and training approach, and that stronger English initialization does not guarantee better transfer efficiency. Adding a single parallel language to the training data yields the largest boost, with additional languages providing smaller but consistent gains.
arXiv AI
Jul 28
ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluation
arXiv:2607. 23058v1 Announce Type: cross Abstract: Multilingual reasoning evaluation overwhelmingly relies on translating English benchmarks, a practice that introduces linguistic artifacts and fails to test culturally-grounded reasoning.
arXiv AI
Aug 5
MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning
arXiv:2608. 03882v1 Announce Type: cross Abstract: Geospatial reasoning, i.