arXiv Machine Learning

h1: A Taste of the Bitter Lesson in Automated Theorem Proving

The paper introduces h1, a simple agentic loop that interfaces a language model with Lean 4, avoiding complex human-designed workflows. h1 achieves state‑of‑the‑art performance on benchmarks such as PutnamBench, FATE‑X, and FormalConjectures100‑Solved, solving eight open problems and matching or surpassing specialized systems like Goedel‑Architect and AlphaProof Nexus. Experiments show that an efficient Lean interface is crucial for cost and success, suggesting that future evaluations should prioritize execution feedback over elaborate workflows.

arXiv AI

Jul 1

Beyond the Library: An Agentic Framework for Autoformalizing Research Mathematics

arXiv:2606. 31134v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated exceptional capabilities in mathematical reasoning, they frequently produce subtle errors that evade human detection.

By Arshia Soltani Moakhar, Iman Gholami, Max Springer, Mahdi JafariRaviz, MohammadTaghi Hajiaghayi

llms agents fine-tuning

arXiv AI

Jun 3

LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks

arXiv:2606. 03303v1 Announce Type: new Abstract: Large Language Models (LLMs) exhibit strong informal mathematical reasoning but struggle to generate mechanically verifiable proofs in formal languages like Lean.

By Po-Nien Kung, Linfeng Song, Dawsen Hwang, Jinsung Yoon, Chun-Liang Li, Simone Severini, Mirek Ol\v{s}\'ak, Edward Lockhart, Quoc V Le, Burak Gokturk, Thang Luong, Tomas Pfister, Nanyun Peng

llms agents benchmarks

arXiv AI

Jun 16

SorryDB: Can AI Provers Complete Real-World Lean Theorems?

arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.

By Austin Letson, Leopoldo Sarra, Auguste Poiroux, Oliver Dressler, Paul Lezeau, Dhyan Aranha, Frederick Pu, Aaron Hill, Miguel Corredera Hidalgo, Julian Berman, George Tsoukalas, Lenny Taelman

llms agents benchmarks

arXiv AI

Jul 21

Self-Modifying Lean Proof Agents with Verifier-Grounded Benchmark Coevolution

arXiv:2607. 17352v1 Announce Type: new Abstract: Designing effective Lean proof agents is a central challenge in formal mathematical reasoning.

By Yuqing Li, Zeguan Wu, Yu Gan, Junyu Liu

agents benchmarks

arXiv AI

Jun 6

Goedel-Architect: Streamlining Formal Theorem Proving with Blueprint Generation and Refinement

arXiv:2606. 06468v1 Announce Type: new Abstract: We introduce Goedel-Architect, an agentic framework for formal theorem proving in Lean 4 centered on blueprint generation and refinement.

By Jui-Hui Chung, Ziyang Cai, Zihao Li, Qishuo Yin, Rohit Agarwal, Simon Park, Rodrigo Porto, Narutatsu Ri, Ziran Yang, Shange Tang, Xingyu Dang, Hongzhou Lin, Mengdi Wang, Danqi Chen, Chi Jin, Liam H Fowl, Sanjeev Arora

agents benchmarks

arXiv AI

Oct 15

Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean

arXiv:2610.17331v1 Announce Type: new Abstract: General-purpose coding agents use standard shell and file interactions, yet formal theorem-proving agents often rely on specialized, ad-hoc tooling. Th...

By Tomas Ortega, Simon Park, Ayush Khaitan, Liam Fowl, Alex Kontorovich, Sanjeev Arora

llms agents benchmarks

arXiv AI

Jul 17

MathCoPilot: An Interactive System for Human-AI Symbiotic Paradigm of Mathematical Research

arXiv:2607. 14582v1 Announce Type: new Abstract: Existing LLM-based theorem provers have achieved impressive results on formal mathematics benchmarks, yet they remain confined to acting as autonomous agents that prove a stated proposition.

By Junjie Zhang, Jiayu Liu, Wenbin Liu, Zhenya Huang, Doudou Wang, Yan Jiang, Leiye Xu, Tao Xiong, Wen Huang, Qi Liu, Guoping Hu, Enhong Chen, Mengping Zhang, Xiangdong Ye

llms rag agents benchmarks

arXiv AI

Jun 8

Lean4Agent: Formal Modeling and Verification for Agent Workflow and Trajectory

arXiv:2606. 06523v1 Announce Type: new Abstract: Equipping Large Language Models (LLMs) to execute reliable multi-step workflows has become a central challenge in artificial intelligence.

By Ruida Wang, Jerry Huang, Pengcheng Wang, Xuanqing Liu, Luyang Kong, Tong Zhang

llms agents

arXiv AI

Oct 2

RMA: Context-Orchestrated Research Math Agents

arXiv:2605.22875v2 Announce Type: replace Abstract: Long-horizon mathematical reasoning fails less often because a model cannot produce a valid next step than because an agent fails to maintain and e...

By Zelin Zhao, Bo Yuan, Yuchen Zhu, Jaemoo Choi, Yongxin Chen

llms agents benchmarks

arXiv AI

Aug 11

P$^{3}$: Joint Program-and-Proof Planning for Verified Code Generation

arXiv:2608. 09277v1 Announce Type: new Abstract: Verified code generation asks a large language model (LLM) to generate both an executable program and a machine-checkable proof that the program meets a formal specification, promising software that is correct by construction.

By Zenan Li, Ziran Yang, Peiyang Song, Zhaoyu Li, Kaiyu Yang

llms agents benchmarks

arXiv AI

Jun 30

LAMP: Lean-based Agentic framework with MCP and Proof Repair

arXiv:2606. 28841v1 Announce Type: cross Abstract: Large language models are increasingly capable of mathematical reasoning, but the proofs they generate are often unreliable and hard to verify.

By Santhana Srinivasan R, Maithilee Patawar

llms agents fine-tuning benchmarks

arXiv AI

Jun 9

Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery

arXiv:2606. 08728v1 Announce Type: new Abstract: Mathematical reasoning has long served as a stringent test of machine intelligence; over the past decade, it has moved from a niche problem within NLP to one of the most consequential AI frontiers.

By Syed Rifat Raiyan, Mohsinul Kabir, Hasan Mahmud, Md Kamrul Hasan

llms agents nlp multimodal benchmarks

Related stories