arXiv Machine Learning
Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 15
Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR
The paper surveys 58 studies on verifier errors in reinforcement learning with verifiable rewards (RLVR), examining how such errors affect training and which mitigations are effective. It finds that the type of error—rather than its frequency—has a larger impact on learning dynamics, with random errors mainly slowing progress and false positives potentially causing learning plateaus or collapse. The authors compile findings into comparison tables, reconcile conflicting reports, and propose a reporting protocol to standardize future research.
arXiv AI
Oct 5
Counterexample Generation via Per-Theorem Symbolic Verifiers: When Imitation Hurts and Reinforcement Repairs
The paper introduces SymCE, a dataset of 4,707 false undergraduate‑algebra and real‑analysis conjectures each paired with a deterministic Python verifier that can be executed to check truth. It studies counterexample generation by training a 4B‑parameter language model (Qwen3‑4B) first with supervised fine‑tuning and then with reinforcement learning (GRPO) using the verifier as a reward. The results show that counterexample‑only fine‑tuning can destroy true‑theorem recognition, while reinforcement learning with a sparse outcome‑only reward restores and surpasses baseline performance, achieving higher success rates than larger open‑weight models and competitive performance on other math benchmarks. "whyItMatters":"The study demonstrates that reinforcement learning guided by a per‑theorem verifier can effectively repair the shortcomings of supervised fine‑tuning in theorem proving, offering a practical approach to improve language models’ mathematical reasoning capabilities."
arXiv AI
Aug 18
Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR
arXiv:2605. 02909v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Sep 10
Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts
The paper investigates whether verifier errors are independent within groups of completions generated by the Qwen2.5-1.5B model on benchmark datasets. Analyses of 24,998 groups of eight completions reveal a pooled within‑group verifier‑error correlation of 0.530, indicating significant clustering of errors. The degree of dependence varies by answer form, with fractions, radicals, symbolic expressions, and intervals showing stronger clustering than unit annotations and percent signs, and up to 0.83% of groups exhibit disagreement in advantage signs across rule‑based verifier configurations.
arXiv Machine Learning
Oct 15
Verifier-Grounded Agentic RL for Provably Correct Code
arXiv:2610.19298v1 Announce Type: new Abstract: Reinforcement learning for code usually rewards passing tests, which a model can do while remaining wrong on the inputs the tests do not cover. A progr...
arXiv AI
Jun 30
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.