KAPPA is an open‑source toolkit that uses language‑model agents to translate LaTeX proofs from statistical papers into Lean 4, where each numbered result is verified step by step against the original text. The system assigns one of four verdicts—verified unconditionally, verified conditionally, issue recorded, or unfinished—to every result, and it can run unattended, re‑check only changed sections, and report resource usage. In tests on four example papers and three published statistical papers, KAPPA detected the known errors and additional issues that were later confirmed by hand, while a direct language‑model approach without the kit produced fewer correct verdicts and sometimes over‑claimed proofs.
By Zihua She, Xiao Wang
PaperFormalizer is a Lean-based pipeline that assists authors in identifying and closing gaps in their research papers, converting them into Lean-verified versions. Starting from LaTeX, it transcribes the paper into a dependency graph, audits statements for faithfulness and correctness, and formalizes them in Lean using Mathlib and optional domain libraries. The process is author-in-the-loop, prompting the author whenever multiple interpretations exist and allowing them to accept, decline, or redirect proposed repairs, ultimately producing an annotated paper, a change report, and linked Lean code.
By Jiyuan Tan, Vasilis Syrgkanis, Jose Blanchet
SemantEq introduces a principled method for verifying that formal statements faithfully translate natural‑language mathematics by requiring each statement to transform into the other through a chain of allowed logical steps, rather than relying on trivial logical equivalence. The system evaluates proofs using Lean’s proof terms, achieving 100 % precision and 65.6 % recall on the ProofNetVerif test split, outperforming prior methods. A complementary audit tool, SemantEq‑ProofAudit, further improves recall for equivalent pairs while rejecting all non‑equivalent pairs.
By Xinrui Li, Pawan Sasanka Ammanamanchi, Siddharth Bhat, Rianna Jamdar, Yongxi Lin, Stella Biderman
arXiv:2610.17358v1 Announce Type: cross
Abstract: Formal verification proves that a program satisfies its specification, but says nothing about whether the specification captures what the author mean...
By Martin Amster, One An, Holly Baker, Fernanda Graciolli, Simon Henniger, Nada Amin
arXiv:2609.39228v1 Announce Type: new
Abstract: We present FYAN, a human--AI harness for document-level mathematical formalization. Rather than treating theorems in isolation, FYAN coordinates an end...
By Wei Zhao, Yangshuo Zou, Chengxiang Ding, Yifan Wu, Xuchuan Wang, Zimu Mao, Lei Zhang, Tao Luo
arXiv:2607. 22511v1 Announce Type: cross Abstract: Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation.
By Jiyuan Tan, Vasilis Syrgkanis
CausalSmith is a framework that automates theoretical research in causal inference by integrating a Lean proof assistant with a self‑improving agentic pipeline. It uses Causalean, a Lean library of over 7,000 machine‑checked declarations, and a pipeline that selects topics, proposes results, formalizes statements, constructs proofs, and audits them against informal claims. The system’s artifacts and source code are publicly available on GitHub.
By Jiyuan Tan, Vasilis Syrgkanis
ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.
By Mushir Akhtar
arXiv:2605.22875v2 Announce Type: replace
Abstract: Long-horizon mathematical reasoning fails less often because a model cannot produce a valid next step than because an agent fails to maintain and e...
By Zelin Zhao, Bo Yuan, Yuchen Zhu, Jaemoo Choi, Yongxin Chen
The paper examines how AI‑written research papers often contain misattributed numerical claims. By re‑analyzing 540 judgments from a benchmark of nine AI agents on fifteen research problems, the authors manually labeled 256 claims and found that most failures stem from misalignment between the paper and the underlying work rather than outright cheating. They propose a deterministic claim‑to‑evidence checker that achieves over 90% precision and, when combined with an LLM verifier, offers a more reliable audit of AI‑generated research.
By Mosab Alfaqeeh, Omar Hourani, Muath Hourani, Hashem Alfaqeeh
ProofJudge is an agentic large language model that evaluates the quality of formal proofs in Lean 4 beyond mere correctness. It scores proofs on five dimensions—library leverage, automation fit, structural clarity, statement quality, and Mathlib conventions—using tool access to the relevant repository commit. The system was tested on 218 Mathlib declarations, achieving alignment with human reviewers between 63.5% and 80.8% and releasing its harness, dataset, and traces for open research.
By Shane Caldwell
Formal proofs in Lean 4 that pass the kernel's type checker can nonetheless vary widely in quality. We introduce ProofJudge, an agentic LLM-as-judge system that scores formal proof quality along five...