arXiv AI By Xiyuan Zhou, Zhuoqi Li, Xinlei Wang, Yirui He, Yuhao Wu, Yuheng Cheng, Yan Xu, Junhua Zhao, Jinjin Gu

RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving

Read the original on arXiv AI →

RePro is a framework that rewrites benchmark problems for large language models (LLMs) in mathematical problem solving, ensuring that the rewritten problems and their answers are valid and correct through Lean-verified proofs. It integrates Lean-oriented neural automated theorem provers (ATPs) to regenerate answers, achieving 100% well-definedness, feasibility, and answer correctness on GSM8K and MATH datasets. Experiments show that models’ performance drops on these proof‑verified rewritten benchmarks, indicating sensitivity to surface‑level and structural variations and potential memorization effects.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
1d ago

Compiler-Guided Adaptive Proof Search with Cross-Model Synergy on Context-Dependent Theorem Proving

The paper introduces a compiler-guided adaptive proof search framework for Lean 4 theorem proving, addressing the challenge of context-dependent proofs in real-world projects. It balances exploration and exploitation by generating diverse starting points via dual-model generation and resampling when stagnation occurs, while refining promising states using compiler-grounded pairwise comparison. Experiments on seven Lean 4 projects from miniCTX‑v2 demonstrate that the method improves average pass rates by 12.8 percentage points within a pass@32 budget and reduces LLM calls by 21.9 % compared to pass@k baselines.

By Zhuo Liu, Ding Yu, Hangfeng He
arXiv AI
Jun 16

Formalize Once, Edit the Rest: Efficient Lean-Based Answer Selection for Math Reasoning

arXiv:2606. 15972v1 Announce Type: cross Abstract: With large language models (LLMs) increasingly applied to mathematical reasoning, formal proof assistants such as Lean can be leveraged to verify reasoning outputs with machine-checkable rigor, enabling use cases such as answer selection in test-time scaling with K sampled candidate answers.

By Ji Feng, Zhouxing Shi
arXiv AI
Aug 28

FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence

FaithSieve is a Lean‑assisted framework that fine‑grains natural‑language mathematical proofs into local reasoning units, extracts typed proof obligations, and verifies them with formal evidence gated by semantic alignment. It introduces two expert‑verified datasets—ProofLoc‑Olympiad and ProofLoc‑University—to benchmark first‑error localization. On these benchmarks, FaithSieve outperforms direct‑judging baselines, achieving 81.43% and 84.5% exact first‑error accuracy respectively.

By Ziyu Wang, Qiming Dai, Yishan Wu, Zaiwen Wen