LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to capture how models behave on longer, more dependency-rich mathematical developments.
arXiv:2606. 09450v1 Announce Type: new Abstract: LLMs have recently achieved strong results on formal proving benchmarks.
By QuocViet Pham, Elvir Karimov, Andrey Galichin, Ivan Oseledets
ProofEvolve is a neuro‑symbolic framework that evolves formally verified symbolic proof structures alongside neural models to expand the knowledge boundary in automated theorem proving. The neural component proposes variation operators such as decompositions, repairs, and schema recombinations, while the Lean kernel verifies every proof transition, ensuring formal soundness. Across three competition‑level Lean benchmarks, ProofEvolve achieves the highest average solve rate among evaluated proof systems.
By Wenqian Ye, Ziwei Guan, Eric Xie, Bohan Liu, Shivani Modi, Buyun Zhang, Ellie Dingqiao Wen, Henry Kautz, Aidong Zhang
arXiv:2506. 17104v2 Announce Type: replace Abstract: Large language models (LLMs) have shown promising first-order logic (FOL) reasoning capabilities with applications in various areas.
By Chuxue Cao, Mengze Li, Juntao Dai, Jinluan Yang, Zijian Zhao, Shengyu Zhang, Weijie Shi, Chengzhong Liu, Sirui Han, Yike Guo
arXiv:2606. 12594v1 Announce Type: new Abstract: Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning traces of formal proof search, making both supervised fine-tuning (SFT) and sampling expensive.
By Joshua Ong Jun Leang, Zheng Zhao, Mihaela C\u{a}t\u{a}lina Stoian, Qiyuan Xu, Haonan Li, Wenda Li, Shay B. Cohen, Eleonora Giunchiglia
MathAdv is a diagnostic benchmark for formal theorem proving that covers 13 undergraduate- and graduate-level mathematics domains. It includes Lean 4 proofs and up to three auxiliary tasks—multiple-choice questions, fill-in-the-blank problems, and expert-crafted transformations—to probe knowledge, informal reasoning, and robustness to problem presentation. Evaluation of current theorem provers shows formalization is a major bottleneck, performance varies by domain, natural-language guidance can help or hinder models, and equivalent reformulations reveal significant robustness gaps.
By Jiaxin Yuan, Connor Martinez Lockhart, Xiaoyu Liu, Jiaqi Wang, Chenghao Deng, Xiayimei Han, Vlasios Mastrantonis, Dmitrii Gudin, Shaopeng Zhu, Abdirisak Abdullahi Mohamed, Bilal Hamdi Aytekin, Jiewen Lang, Zezheng Song, Furong Huang