arXiv AI
Jun 9
TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
arXiv:2606. 09450v1 Announce Type: new Abstract: LLMs have recently achieved strong results on formal proving benchmarks.
TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to capture how models behave on longer, more dependency-rich mathematical developments.
Related stories
arXiv AI
Jun 12
Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation
arXiv:2606. 12594v1 Announce Type: new Abstract: Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning traces of formal proof search, making both supervised fine-tuning (SFT) and sampling expensive.
arXiv Machine Learning
Jun 25
Discovering New Theorems via LLMs with In-Context Proof Learning in Lean
arXiv:2509. 14274v3 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated significant promise in formal theorem proving.
arXiv AI
Jun 16
SorryDB: Can AI Provers Complete Real-World Lean Theorems?
arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.
arXiv AI
Jun 2
Formally Solving Answer-Construction Problems in Lean
arXiv:2505. 18492v5 Announce Type: replace Abstract: Mathematical competition problems fall into two broad types: theorem proving, which asks for a proof of a given statement, and answer construction, which requires constructing a property-satifying object with proofs.
arXiv AI
Jun 3
LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks
arXiv:2606. 03303v1 Announce Type: new Abstract: Large Language Models (LLMs) exhibit strong informal mathematical reasoning but struggle to generate mechanically verifiable proofs in formal languages like Lean.
arXiv AI
Jun 16
Towards Advanced Mathematical Reasoning for LLMs via First-Order Logic Theorem Proving
arXiv:2506. 17104v2 Announce Type: replace Abstract: Large language models (LLMs) have shown promising first-order logic (FOL) reasoning capabilities with applications in various areas.
arXiv AI
Jun 16
Mask-Proof: An LLM-based Automated Data Curation Pipeline on Mathematical Proofs
arXiv:2606. 15258v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly capable of mathematical problem solving and can even assist with research-level proofs, yet we still lack a scalable and reproducible way to measure step-level reasoning in long proofs across diverse sources.
Hugging Face Trending Papers
Jul 13
AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
Large language models (LLMs) have achieved remarkable performance on high-school and olympiad-style mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and often rely on final-answer correctness or coarse judgments, leaving the validity of the reasoning process inadequately assessed.
arXiv AI
Jun 3
Proof-Refactor: Refactoring Generated Formal Proofs into Modular Artifacts
arXiv:2606. 03743v1 Announce Type: new Abstract: While Large Language Models (LLMs) have shown strong performance in generating formal proofs, their outputs often remain less readable, modular, maintainable, and reusable than proofs in mature formal mathematics libraries.
arXiv AI
Jun 30
Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving
arXiv:2606. 29493v1 Announce Type: new Abstract: Benchmarks for LLM-assisted theorem proving in Lean are often treated as intrinsically reliable because every solved instance comes with a machine-checked proof.
arXiv AI
Jun 15
MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis
arXiv:2606. 13782v1 Announce Type: new Abstract: Large Language Models (LLMs) have made notable progress in automated theorem proving, yet existing formal benchmarks remain limited in both mathematical coverage and difficulty.