arXiv AI

ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark

arXiv:2505. 23851v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly applied to symbolic mathematics, yet existing evaluations often conflate pattern memorization with genuine reasoning.

arXiv AI
Sep 2

TopoAlign: A Framework for Aligning Code to Math via Topological Decomposition

The paper introduces TopoAlign, a framework that repurposes code repositories to train Math LLMs by decomposing code into docstrings, main functions, and dependency functions and reassembling them into structures that mirror formal mathematical statements. Using this approach, the authors train three state‑of‑the‑art models—DeepSeek‑Math, Qwen‑3, and Herald—and evaluate them on MiniF2F, Putnam, and ProofNet benchmarks. TopoAlign yields significant performance gains, notably a 17.77% improvement on BEq@10 and a 68.82% boost on typecheck@10 for DeepSeek‑Math, while also providing modest gains for Herald.

By Yupei Li, Philipp Borchert, Gerasimos Lampouras
arXiv AI
Aug 18

Euclid-Omni : A Unified Neuro-Symbolic Framework for Plane Geometry

Euclid-Omni is a unified neuro‑symbolic framework that integrates a formal geometry system with Large Language Models and Vision‑Language Models to solve both calculation and proving problems in Euclidean geometry up to Olympiad level. Its core component, Euclidea, automatically generates deductive reasoning steps and algebraic computations, while a data‑generation pipeline creates synthetic symbolic problems, diagrams, and natural‑language translations for training. Experiments show that VLMs trained on this synthetic data outperform on calculation tasks, and LLMs paired with Euclidea match state‑of‑the‑art proving systems using far less compute and data.

By Zhaoyu Li, Hangrui Bi, Youyuan Zhang, Wenjie Ma, Zenan Li, Zhaolei Zhang, Xujie Si, Kaiyu Yang
Hugging Face Trending Papers
Aug 13

Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning.

arXiv Machine Learning
Sep 11

Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification

The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.

By Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen, Eliot Hodges, Dulhan Jayalath, Charles London, Kalyan Ramakrishnan, Jakob Foerster, Cheng Zhang, Flaviu Cipcigan, Philip Torr, Alessandro Abate
arXiv AI
Jun 16

SorryDB: Can AI Provers Complete Real-World Lean Theorems?

arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.

By Austin Letson, Leopoldo Sarra, Auguste Poiroux, Oliver Dressler, Paul Lezeau, Dhyan Aranha, Frederick Pu, Aaron Hill, Miguel Corredera Hidalgo, Julian Berman, George Tsoukalas, Lenny Taelman