arXiv AI

Math Reasoning in LLMs is Organized by Approach, Not Topic

The paper argues that large language models (LLMs) organize their internal mathematical reasoning by reusable reasoning approaches rather than by the benchmark topics they are tested on. Using a generation‑replay protocol, the authors extract activation‑importance signatures from eight models across five math sources, cluster these signatures, and find that the resulting groups align more closely with reasoning approaches than with topics. The study shows that changing the requested reasoning approach shifts cluster assignments, while paraphrasing the prompt does not, underscoring the primacy of approach over topic in LLM reasoning.

arXiv AI

2d ago

LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches

LiveMathematicianBench is a dynamic multiple‑choice benchmark for research‑level mathematical reasoning, built from recent arXiv papers published after model training cutoffs. It introduces a thirteen‑category logical taxonomy of theorem types and uses a proof‑sketch‑guided distractor pipeline to create plausible but invalid answer choices, enhancing sensitivity to genuine reasoning. Evaluation shows current large language models perform poorly, with the best model scoring 43.5% overall and only 17.6% under substitution‑resistant conditions, indicating the benchmark’s difficulty and realism.

By Linyang He, Qiyao Yu, Hanze Dong, Baohao Liao, Xinxing Xu, Micah Goldblum, Jiang Bian, Nima Mesgarani

llms benchmarks

arXiv AI

Jul 8

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

arXiv:2605. 19723v2 Announce Type: replace-cross Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems.

By Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima

llms fine-tuning benchmarks

arXiv AI

Jun 8

A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning

arXiv:2606. 07410v1 Announce Type: cross Abstract: The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning.

By Yuxiang Chen, Jun Wang

llms

arXiv AI

Jun 10

RealMath-Eval: Why SOTA Judges Struggle with Real Human Reasoning

arXiv:2606. 10254v1 Announce Type: new Abstract: While Large Language Models (LLMs) have achieved near-perfect performance in \emph{solving} high-school mathematics, their ability to \emph{evaluate} the diverse reasoning processes of real human students remains under-examined.

By Yiteng Mao, Kenan Xu, Yijia Lyu, Wenhao Li, Jianlong Chen, Xiangfeng Wang

llms rag benchmarks

arXiv AI

Jul 8

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.

By Daryna Dementieva, Nikolay Babakov, Kathy H\"ammerl, Ilseyar Alimova, Jind\v{r}ich Libovick\'y, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan \"Ozer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser

llms benchmarks

arXiv AI

Jun 10

Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning

arXiv:2606. 09873v1 Announce Type: cross Abstract: Reasoning models achieve strong performance on challenging tasks by generating explicit intermediate reasoning traces before producing a final answer.

By Aditya Sharma, Christopher J. Pal, Amal Zouaq

llms rag multimodal benchmarks

arXiv AI

Jul 21

Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering

arXiv:2607. 18100v1 Announce Type: new Abstract: Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable.

By Sheldon Yu, Tong Yu, Xunyi Jiang, Rohan Surana, Gagan Mundada, Sungchul Kim, Lina Yao, Julian McAuley, Junda Wu

llms

arXiv AI

Jul 9

MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

arXiv:2607. 07391v1 Announce Type: new Abstract: Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue.

By Charbel Al Bateh, Samer Saab Jr

llms benchmarks

arXiv Computation and Language

4d ago

Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

The paper demonstrates that frontier language models can be prompted to expose their internal chain-of-thought reasoning via a simple custom tool. By comparing these extracted traces to native reasoning on open-source models, the authors confirm that the externalized reasoning aligns with genuine reasoning and outperforms no-reasoning baselines across math, science, and code tasks. They further analyze the structure of the reasoning, noting token-efficient, directed reasoning in models like GPT‑6 Astra, which externalizes only crucial steps while handling elementary ones internally.

By Xiaoyu Luo, Tao Ren, Wenrui Yu, Xiao Li, Qiongxiu Li, Johannes Bjerva

llms benchmarks

arXiv AI

Sep 1

SHAPE of Chain-of-Thought in Math Reasoning

arXiv:2608.28600v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their...

By Jonghyun Song, Sangjun Song, Minjae Oh, Haesung Pyun, Sungsik Lee, Yohan Jo

llms reinforcement-learning benchmarks

arXiv AI

Jul 22

Robust Reasoning Benchmark

arXiv:2604. 08571v3 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) achieve high performance on standard mathematical benchmarks, their problem-solving abilities depend on the context and textual formatting.

By Pavel Golikov, Evgenii Opryshko, Gennady Pekhimenko, Mark C. Jeffrey

llms nlp benchmarks

arXiv AI

Jul 24

Representation Robustness Under Executable Reasoning Constraints in Large Language Models for Mathematical Problem Solving

arXiv:2607. 20520v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated on mathematical problem solving, yet prior work often treats representationally equivalent formulations as interchangeable and conflates reasoning errors with interface failures.

By Sagnik Nath, Edith Aurora Graf, Liang Zhang, Diego Zapata-Rivera

llms

Related stories