SorryDB: Can AI Provers Complete Real-World Lean Theorems?
arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.
arXiv:2602. 18767v3 Announce Type: replace-cross Abstract: In Machine-Assisted Theorem Proving, a theorem proving agent searches for a sequence of expressions and tactics that can prove a statement in a proof assistant.
arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.
ProofEvolve is a neuro‑symbolic framework that evolves formally verified symbolic proof structures alongside neural models to expand the knowledge boundary in automated theorem proving. The neural component proposes variation operators such as decompositions, repairs, and schema recombinations, while the Lean kernel verifies every proof transition, ensuring formal soundness. Across three competition‑level Lean benchmarks, ProofEvolve achieves the highest average solve rate among evaluated proof systems.
NanoProof is the first factorized execution‑guided theorem prover in Lean 4 whose training data, tooling, pipeline, and weights are all publicly released, enabling end‑to‑end reproducibility with open‑source resources. The authors provide a dataset of structured proof trees and a programmatic interaction tool for Lean 4, emphasizing compute efficiency to make training and evaluation more accessible. NanoProof achieves a 50.8% pass@16 on MiniF2F‑Test, outperforming comparable systems while using far less compute, demonstrating that such provers can be rebuilt from scratch with modest resources.
Ax-Prover is a multi‑agent framework that uses Large Language Models (LLMs) combined with Lean tools via the Model Context Protocol (MCP) to generate formally correct proofs in mathematics and quantum physics. It can operate autonomously or collaboratively with human experts, and its performance on public math benchmarks and new Lean benchmarks in abstract algebra and quantum theory shows it is competitive with state‑of‑the‑art provers and often outperforms them. The system also demonstrated practical assistant capabilities by helping a mathematician formalize a complex cryptography theorem.
The paper introduces EasyProver, a Lean theorem prover that uses recursive agents, isolated contexts, asynchronous search, parallel tactic probing, and an elastic elaboration service. It achieves near‑saturation results on several benchmark suites, proving 669/672 PutnamBench problems, all six IMO 2026 formalizations, 597/600 NTP4VC verification conditions, and all 189 VERINA tasks. Additionally, the authors present Aegis, a pipeline that transforms competitive programming problems into harder verification tasks by generating safe Rust solutions, translating them to Lean, and producing 2,045 Rust‑Lean program pairs with 11,533 candidate specifications, enabling new benchmarks for provers and reinforcement learning.
arXiv:2606. 09450v1 Announce Type: new Abstract: LLMs have recently achieved strong results on formal proving benchmarks.
LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to capture how models behave on longer, more dependency-rich mathematical developments.
arXiv:2610.17697v1 Announce Type: cross Abstract: AI systems now write long and difficult mathematical proofs, and checking their step-by-step validity is becoming a bottleneck. A learned verifier co...
RePro is a framework that rewrites benchmark problems for large language models (LLMs) in mathematical problem solving, ensuring that the rewritten problems and their answers are valid and correct through Lean-verified proofs. It integrates Lean-oriented neural automated theorem provers (ATPs) to regenerate answers, achieving 100% well-definedness, feasibility, and answer correctness on GSM8K and MATH datasets. Experiments show that models’ performance drops on these proof‑verified rewritten benchmarks, indicating sensitivity to surface‑level and structural variations and potential memorization effects.
arXiv:2607. 09217v1 Announce Type: new Abstract: In this system paper, we present OpenProver, an open-source system for LLM-driven automated theorem proving (ATP) with integrated Lean 4 formal verification.
TreeThink is an open‑source Python library that provides modular, fully asynchronous tree search for neural theorem proving. It integrates established tree‑search methods with vLLM inference pipelines and supports a range of node evaluation techniques, from lightweight heuristics to neural evaluators. The library connects directly to the REPL servers of Lean 4, Rocq, and Isabelle/HOL, enabling real‑time verification and proof‑state extraction, and it has been evaluated on miniF2F and MATH500, achieving up to an 8.0× wall‑clock speedup from asynchronous execution.
arXiv:2606. 12594v1 Announce Type: new Abstract: Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning traces of formal proof search, making both supervised fine-tuning (SFT) and sampling expensive.