arXiv AI By Aravind Mandiga, Guoming Li

How Stable Are LLM-Generated Proofs Under Program Refactoring? A Stress Test for Verifiable Code Generation

Read the original on arXiv AI →

The paper introduces RefactorProof, a benchmark that tests whether proofs generated for code remain valid after program refactoring that preserves behavior. Using 458 Lean-certified variants derived from 185 VERINA tasks, the study finds that structural refactorings expose brittleness in proofs, with task-normalized structural PSR ranging from 4.3% for supplied references to 13.3–26.8% for specialized provers. Despite higher failure rates, specialized prover proofs (DeepSeek-Prover-V2-7B, Goedel-Prover-V2-32B, Goedel-Prover-V2-8B) show significantly higher unchanged-proof survival compared to reference artifacts, and many failures can be repaired with minimal edits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 19

Repair, Retry, and the Cost of Hindsight: Verifier Feedback in Formal Theorem Proving

The paper investigates how verifier feedback influences self‑repair in formal theorem proving. Using controlled one‑step experiments on Lean with frozen Qwen3.6‑27B and DeepSeek‑Prover‑V2‑7B, the authors find that showing a model its failed predecessor can hurt success and increase copying, while providing the corresponding compiler error can partially recover performance. In sequential trials, Qwen benefits from correct diagnostics within failed‑proof histories, but no feedback condition outperforms simple instruction‑only resampling, indicating that verifier feedback can aid repair yet cannot fully offset the cost of conditioning on a failed attempt.

By Jack Li, Sophia Li
arXiv Computation and Language
Oct 16

Reasoning without Gold Standards: A Proxy-Judge Theory of Autoformalization

The paper introduces a reference‑free proxy‑judge framework for autoformalization, separating correctness into grounding, statement faithfulness, coverage, and inference. It presents three theoretical results linking proxy error rates to pass‑but‑wrong rates, the impact of false rejections on refinement, and a decomposition of final pass rates. The framework is instantiated with eight proxies and evaluated on ProofNet, miniF2F, e‑SNLI, and ProntoQA, showing that the predictions hold and identifying failure routes.

By Lei Xu, Xin Quan, Andr\'e Freitas