FaithSieve is a Lean‑assisted framework that fine‑grains natural‑language mathematical proofs into local reasoning units, extracts typed proof obligations, and verifies them with formal evidence gated by semantic alignment. It introduces two expert‑verified datasets—ProofLoc‑Olympiad and ProofLoc‑University—to benchmark first‑error localization. On these benchmarks, FaithSieve outperforms direct‑judging baselines, achieving 81.43% and 84.5% exact first‑error accuracy respectively.
By Ziyu Wang, Qiming Dai, Yishan Wu, Zaiwen Wen
arXiv:2606. 21678v2 Announce Type: replace-cross Abstract: Language models can generate plausible rationales for their predictions, but these explanations may not faithfully represent the model's internal reasoning.
By Vatsal Ananthula, Adarsh Kumarappan
The paper introduces an epistemically and formally grounded ensemble (EFG) of large language model judges to evaluate autoformalization tasks in formal mathematics. It defines four criteria—logical preservation, mathematical consistency, formal quality, and formal validity—to provide a transparent, multi‑granular assessment. Experiments show that this ensemble outperforms coarse‑grained models, offering a scalable and interpretable proxy for evaluating formal mathematical reasoning.
By Lan Zhang, Marco Valentino, Jordan Meadows, Andre Freitas
arXiv:2609.35790v1 Announce Type: cross
Abstract: While neural theorem provers have achieved impressive milestones in formal mathematics, they largely operate on the assumption that faithful Lean 4 f...
By Thomas Hirtz, Farzad Jafarrahmani, Abdelmouksit Sagueni, Xiang Zhou, Wengping Deng, Liang Zhang
arXiv:2606. 14867v1 Announce Type: cross Abstract: Proof autoformalization aims to translate a mathematical informal proof written in natural language into a formal proof in a formal language such as Lean~4.
By Zhengtao Gui, Sheng Yang, Zhouxing Shi
arXiv:2607. 26102v1 Announce Type: cross Abstract: Mathematical chain of thought (CoT) evaluation is commonly reduced to whether the final answer matches a reference.
By Vivek Shukla, Varun Shukla, Atul, Divya Mishra, Mehul Kumar Das
arXiv:2608. 14221v1 Announce Type: new Abstract: Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4.
By Lushi Pu, Weiming Zhang, Xinheng Xie, Zixuan Fu, Bingxiang He, Hengyu Zhao, Hongya Lyu, Xin Li, Jie Zhou, Yudong Wang
GRACE is a step‑level benchmark for evaluating the faithfulness of chain‑of‑thought reasoning over context. It provides human annotations for each step in CoT traces from 10 models across 4 datasets, labeling faithfulness, error category, and natural‑language explanations. The benchmark introduces a data‑driven taxonomy that splits errors into GRACE‑Inference (deductive) and GRACE‑Grounding (factual) tracks, each with four categories, and demonstrates that incorporating step‑level faithfulness signals can improve downstream accuracy and reasoning reliability.
By Hoang Pham, Dong Le, Anh Tuan Luu
Magenta is a training‑free pipeline that bridges informal natural‑language mathematical problems and formal Lean 4 verification. Given a problem in plain text, it generates an answer, translates it into a Lean 4 statement, and constructs a machine‑checked proof. The system includes a statement judge to ensure the formalisation matches the original problem and an error‑attribution judge to guide corrections, achieving perfect accuracy on olympiad benchmarks and solving all six IMO 2026 problems when combined with K2‑Horizon‑7B.
By Joshua Ong Jun Leang, Haonan Li, Zheng Zhao, Xinyi Shang, Wenda Li, Zhengzhong Liu, Erix Xing, Shay Cohen, Eleonora Giunchiglia
arXiv:2605. 20531v2 Announce Type: replace-cross Abstract: Reliable verification of proofs remains a bottleneck for training and evaluating AI systems on hard mathematical reasoning.
By Slim Barkallah, Luke Bailey, Kaiyue Wen, Mohammed Abouzaid, Tengyu Ma
arXiv:2607. 19407v1 Announce Type: new Abstract: Formal theorem proving has emerged as a frontier challenge for machine learning, yet the ecosystem is fragmented: proofs remain siloed across incompatible systems, limiting both training data for learning-based provers and the portability of verified results.
By Jiayi Wu, Robert Joseph George, Anima Anandkumar
arXiv:2607. 13303v1 Announce Type: cross Abstract: Formal contracts are essential for software testing and verification, yet writing them remains labor-intensive and error-prone.
By Hongyi Liu, Madhusudan Parthasarathy, Adithya Murali