The paper presents a neural evaluator that predicts Lean 4 theorem‑proving verdicts and compiler feedback, trained on 4.84 million labelled programs. A two‑stage approach—first screening candidates with a validity classifier, then scoring positives with a feedback model—achieves 87.8 % accuracy on a 68,932‑program benchmark and processes 3.5 candidates per second on an H100 GPU, outperforming a 64‑worker Lean baseline. Integrating this evaluator into a hybrid reward system reduces GRPO training steps by about 31 % while maintaining comparable miniF2F pass rates.
arXiv:2610.17697v1 Announce Type: cross Abstract: AI systems now write long and difficult mathematical proofs, and checking their step-by-step validity is becoming a bottleneck. A learned verifier co...
arXiv:2606. 20068v1 Announce Type: new Abstract: While reinforcement learning from verifiable rewards (RLVR) typically has relied on a single binary verification signal, symbolic proof assistants in formal reasoning offer rich, fine-grained structured feedback.
arXiv:2609.38384v1 Announce Type: new Abstract: Verified proof edits offer a natural source of supervision for improving language-model-generated Lean proofs. Yet verification establishes that an edi...
arXiv:2610.18760v1 Announce Type: new Abstract: Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To...
BoostAPR is a three-stage framework that improves automated program repair by using execution-grounded reinforcement learning with dual reward models. The approach first fine‑tunes a model on execution‑verified demonstrations, then trains a sequence‑level assessor and a line‑level credit allocator from execution outcomes, and finally applies PPO optimization where the line‑level model redistributes rewards to critical edit regions. Evaluated on SWE‑Gym and four benchmarks, BoostAPR achieves significant gains, including 40.7% on SWE‑bench Verified and 95.0% on QuixBugs, demonstrating strong cross‑language generalization.
RePro is a framework that rewrites benchmark problems for large language models (LLMs) in mathematical problem solving, ensuring that the rewritten problems and their answers are valid and correct through Lean-verified proofs. It integrates Lean-oriented neural automated theorem provers (ATPs) to regenerate answers, achieving 100% well-definedness, feasibility, and answer correctness on GSM8K and MATH datasets. Experiments show that models’ performance drops on these proof‑verified rewritten benchmarks, indicating sensitivity to surface‑level and structural variations and potential memorization effects.
arXiv:2603. 19329v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can generate plausible code but offer limited guarantees of correctness.
arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
The paper introduces Actor‑Critic with Action Chunking (AC2), a method that assigns credit to short action chunks instead of entire trajectories, enabling policy updates without waiting for terminal rewards. AC2 employs local readiness, reference solutions, and 10k‑token chunks to make critic‑based credit assignment reliable. Experiments on Qwen3‑4B with FineProofs‑RL show AC2 surpasses GRPO’s peak validation score while using 2.5× fewer decoding FLOPs and fewer training steps.
NanoProof is the first factorized execution‑guided theorem prover in Lean 4 whose training data, tooling, pipeline, and weights are all publicly released, enabling end‑to‑end reproducibility with open‑source resources. The authors provide a dataset of structured proof trees and a programmatic interaction tool for Lean 4, emphasizing compute efficiency to make training and evaluation more accessible. NanoProof achieves a 50.8% pass@16 on MiniF2F‑Test, outperforming comparable systems while using far less compute, demonstrating that such provers can be rebuilt from scratch with modest resources.
arXiv:2608. 01804v1 Announce Type: new Abstract: Post-training large language models (LLMs) via reinforcement learning (RL) has significantly advanced code generation capabilities.
arXiv:2606. 03892v1 Announce Type: cross Abstract: Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server's actual state (so the generated tool calls fail to execute), and recall-based RL rewards incentivize verbose tool-calling patterns.