arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.
By Austin Letson, Leopoldo Sarra, Auguste Poiroux, Oliver Dressler, Paul Lezeau, Dhyan Aranha, Frederick Pu, Aaron Hill, Miguel Corredera Hidalgo, Julian Berman, George Tsoukalas, Lenny Taelman
arXiv:2605.22875v2 Announce Type: replace
Abstract: Long-horizon mathematical reasoning fails less often because a model cannot produce a valid next step than because an agent fails to maintain and e...
By Zelin Zhao, Bo Yuan, Yuchen Zhu, Jaemoo Choi, Yongxin Chen
arXiv:2606. 09450v1 Announce Type: new Abstract: LLMs have recently achieved strong results on formal proving benchmarks.
By QuocViet Pham, Elvir Karimov, Andrey Galichin, Ivan Oseledets
NanoProof is the first factorized execution‑guided theorem prover in Lean 4 whose training data, tooling, pipeline, and weights are all publicly released, enabling end‑to‑end reproducibility with open‑source resources. The authors provide a dataset of structured proof trees and a programmatic interaction tool for Lean 4, emphasizing compute efficiency to make training and evaluation more accessible. NanoProof achieves a 50.8% pass@16 on MiniF2F‑Test, outperforming comparable systems while using far less compute, demonstrating that such provers can be rebuilt from scratch with modest resources.
By Mat\v{e}j Kripner, Milan Straka
arXiv:2602.18307v2 Announce Type: replace-cross
Abstract: Large language models have achieved striking results in interactive theorem proving, particularly in Lean. However, most benchmarks for LLM-b...
By Yutong Xin, Qiaochu Chen, Greg Durrett, I\c{s}il Dillig
LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to capture how models behave on longer, more dependency-rich mathematical developments.