FaithSieve is a Lean‑assisted framework that fine‑grains natural‑language mathematical proofs into local reasoning units, extracts typed proof obligations, and verifies them with formal evidence gated by semantic alignment. It introduces two expert‑verified datasets—ProofLoc‑Olympiad and ProofLoc‑University—to benchmark first‑error localization. On these benchmarks, FaithSieve outperforms direct‑judging baselines, achieving 81.43% and 84.5% exact first‑error accuracy respectively.
By Ziyu Wang, Qiming Dai, Yishan Wu, Zaiwen Wen
Large language models (LLMs) have achieved remarkable performance on high-school and olympiad-style mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and often rely on final-answer correctness or coarse judgments, leaving the validity of the reasoning process inadequately assessed.
arXiv:2602.02377v3 Announce Type: replace
Abstract: While large language models (LLMs) have achieved strong performance on mathematical problems with verifiable answers, many advanced problems are pr...
By Haotong Yang, Zitong Wang, Shijia Kang, Siqi Yang, Wenkai Yu, Xu Niu, Yike Sun, Yi Hu, Zhouchen Lin, Muhan Zhang
MathAdv is a diagnostic benchmark for formal theorem proving that covers 13 undergraduate- and graduate-level mathematics domains. It includes Lean 4 proofs and up to three auxiliary tasks—multiple-choice questions, fill-in-the-blank problems, and expert-crafted transformations—to probe knowledge, informal reasoning, and robustness to problem presentation. Evaluation of current theorem provers shows formalization is a major bottleneck, performance varies by domain, natural-language guidance can help or hinder models, and equivalent reformulations reveal significant robustness gaps.
By Jiaxin Yuan, Connor Martinez Lockhart, Xiaoyu Liu, Jiaqi Wang, Chenghao Deng, Xiayimei Han, Vlasios Mastrantonis, Dmitrii Gudin, Shaopeng Zhu, Abdirisak Abdullahi Mohamed, Bilal Hamdi Aytekin, Jiewen Lang, Zezheng Song, Furong Huang
arXiv:2607. 11307v1 Announce Type: new Abstract: Full-proof autoformalization bridges extensive mathematical proofs in natural language with formally validated reasoning, offering a pathway to elevate the ceiling of verifiable mathematical reasoning.
By Tian-Shuo Liu, Shiyuan Zhang, Zijie Geng, Haoyu Liu, Runjie Xu, Pengyuan Wang, Lei Yuan, Yang Yu
arXiv:2606. 12594v1 Announce Type: new Abstract: Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning traces of formal proof search, making both supervised fine-tuning (SFT) and sampling expensive.
By Joshua Ong Jun Leang, Zheng Zhao, Mihaela C\u{a}t\u{a}lina Stoian, Qiyuan Xu, Haonan Li, Wenda Li, Shay B. Cohen, Eleonora Giunchiglia
arXiv:2605. 20531v2 Announce Type: replace-cross Abstract: Reliable verification of proofs remains a bottleneck for training and evaluating AI systems on hard mathematical reasoning.
By Slim Barkallah, Luke Bailey, Kaiyue Wen, Mohammed Abouzaid, Tengyu Ma
AdvancedMathBench is a new benchmark suite that evaluates large language models on advanced mathematical proof generation and verification. It includes ProverBench, with 245 problems from undergraduate to doctoral qualifying‑exam levels, and VerifierBench, which tests models’ ability to judge proof validity using 888 expert‑annotated trajectories. The suite features an automatic verification pipeline trained on expert data, and results show that even state‑of‑the‑art models perform poorly, highlighting a gap between generation and verification skills.
By Lingkai Kong, Zijian Wu, Yuzhe Gu, Haiteng Zhao, Zhouqi Hua, Wenyong Huang, Shuang Sun, Zhicheng Xiong, Xiaotian Zhang, Shuya Zhao, Yan Wang, Disheng Xu, Wenwei Zhang, Kai Chen
LiveMathematicianBench is a dynamic multiple‑choice benchmark for research‑level mathematical reasoning, built from recent arXiv papers published after model training cutoffs. It introduces a thirteen‑category logical taxonomy of theorem types and uses a proof‑sketch‑guided distractor pipeline to create plausible but invalid answer choices, enhancing sensitivity to genuine reasoning. Evaluation shows current large language models perform poorly, with the best model scoring 43.5% overall and only 17.6% under substitution‑resistant conditions, indicating the benchmark’s difficulty and realism.
By Linyang He, Qiyao Yu, Hanze Dong, Baohao Liao, Xinxing Xu, Micah Goldblum, Jiang Bian, Nima Mesgarani
HybridProver is a unified framework that combines whole-proof synthesis and tactic-based generation using proof sketches as an intermediate representation. Implemented in Isabelle/HOL, it employs two 7B-scale LLMs trained on optimized Isabelle datasets. On the miniF2F Isabelle benchmark, HybridProver achieved a 73.8% success rate, surpassing the previous state of the art of 61.9%, and ablation studies examined the effects of dataset quality, training settings, and sampling strategies.
By Jilin Hu, Jianyu Zhang, Yongwang Zhao, Talia Ringer
arXiv:2608. 09538v1 Announce Type: cross Abstract: We introduce TCS-Bench, a benchmark for evaluating Large Language Models (LLMs) on research-level Theoretical Computer Science (TCS) proof generation.
By Vincent Cohen-Addad, Dimitris Paparas, Ernest van Wijland, Max Springer, Julien Canitrot-Paradis, Honghao Lin, David Woodruff, Adarsh Kumarappan, Rajesh Jayaram, Rudrajit Das, Lalit Jain, Ola Svensson, Silvio Lattanzi, Mislav Balunovic, Theophane Weber, Vahab Mirrokni
arXiv:2512. 10187v3 Announce Type: replace Abstract: LLMs excel at reasoning, but validating their steps remains challenging.
By Mantas Baksys, Stefan Zetzsche, Olivier Bouissou, Sean B. Holden