NL2AGBench is a benchmark that evaluates how well large language models can translate English geometry problems into the formal language required by AlphaGeometry’s theorem‑proving engine. The study tests ten state‑of‑the‑art LLMs, comparing executable translation accuracy, syntactic correctness, and error types, and finds a large gap between closed‑source and open‑source models. The authors also propose an error taxonomy and test mitigation strategies such as few‑shot prompting, fine‑tuning, and human‑guided hinting, which improve performance across model families.
By Samuel Xiao, Judy Song, Rory Hu, Ziliang Zong
The paper introduces an epistemically and formally grounded ensemble (EFG) of large language model judges to evaluate autoformalization tasks in formal mathematics. It defines four criteria—logical preservation, mathematical consistency, formal quality, and formal validity—to provide a transparent, multi‑granular assessment. Experiments show that this ensemble outperforms coarse‑grained models, offering a scalable and interpretable proxy for evaluating formal mathematical reasoning.
By Lan Zhang, Marco Valentino, Jordan Meadows, Andre Freitas
arXiv:2512.09874v3 Announce Type: replace-cross
Abstract: Correctly parsing mathematical formulas from PDFs is critical for training large language models and building scientific knowledge bases from...
By Pius Horn, Janis Keuper
arXiv:2606. 16003v1 Announce Type: new Abstract: This work investigates the ability of large language models (LLMs) to generate mathematical equations from scientific texts.
By Yifan Mo, Xiao Fu, Yue Su, Qingyu Meng, Koen Hindriks, Qingzhi Liu, Jiahuan Pei
arXiv:2605. 19723v2 Announce Type: replace-cross Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems.
By Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima
arXiv:2510. 04520v2 Announce Type: replace Abstract: Accurate auto-formalization of theorem statements is essential for advancing automated discovery and verification of research-level mathematics, yet remains a major bottleneck for LLMs due to hallucinations, semantic mismatches, and their inability to synthesize new definitions.
By Hanyu Wang, Ruohan Xie, Yutong Wang, Guoxiong Gao, Xintao Yu, Bin Dong
arXiv:2608. 14221v1 Announce Type: new Abstract: Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4.
By Lushi Pu, Weiming Zhang, Xinheng Xie, Zixuan Fu, Bingxiang He, Hengyu Zhao, Hongya Lyu, Xin Li, Jie Zhou, Yudong Wang
arXiv:2604. 03789v2 Announce Type: replace-cross Abstract: Recent advances in large language models have significantly improved their ability to perform mathematical reasoning, extending from elementary problem solving to increasingly capable performance on research-level problems.
By Haocheng Ju, Guoxiong Gao, Jiedong Jiang, Bin Wu, Zeming Sun, Shurui Liu, Leheng Chen, Yutong Wang, Yuefeng Wang, Zichen Wang, Wanyi He, Peihao Wu, Liang Xiao, Ruochuan Liu, Bryan Dai, Bin Dong
arXiv:2609.36187v1 Announce Type: new
Abstract: Symbolic Regression (SR) is a data-driven method for scientific discovery which searches for interpretable analytical relationships within data. Recent...
By Cristina Rossetti, Anna V. Kononova, Thomas B\"ack, Fei Liu, Niki Van Stein
The paper introduces TopoAlign, a framework that repurposes code repositories to train Math LLMs by decomposing code into docstrings, main functions, and dependency functions and reassembling them into structures that mirror formal mathematical statements. Using this approach, the authors train three state‑of‑the‑art models—DeepSeek‑Math, Qwen‑3, and Herald—and evaluate them on MiniF2F, Putnam, and ProofNet benchmarks. TopoAlign yields significant performance gains, notably a 17.77% improvement on BEq@10 and a 68.82% boost on typecheck@10 for DeepSeek‑Math, while also providing modest gains for Herald.
By Yupei Li, Philipp Borchert, Gerasimos Lampouras
arXiv:2607. 25630v1 Announce Type: cross Abstract: Interdisciplinary research is accelerating, yet scientific papers remain difficult to understand outside their home fields.
By Kyuri Im, Michael F\"arber
arXiv:2606. 15972v1 Announce Type: cross Abstract: With large language models (LLMs) increasingly applied to mathematical reasoning, formal proof assistants such as Lean can be leveraged to verify reasoning outputs with machine-checkable rigor, enabling use cases such as answer selection in test-time scaling with K sampled candidate answers.
By Ji Feng, Zhouxing Shi