arXiv Machine Learning
Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 18
Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR
arXiv:2605. 02909v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Sep 10
Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts
The paper investigates whether verifier errors are independent within groups of completions generated by the Qwen2.5-1.5B model on benchmark datasets. Analyses of 24,998 groups of eight completions reveal a pooled within‑group verifier‑error correlation of 0.530, indicating significant clustering of errors. The degree of dependence varies by answer form, with fractions, radicals, symbolic expressions, and intervals showing stronger clustering than unit annotations and percent signs, and up to 0.83% of groups exhibit disagreement in advantage signs across rule‑based verifier configurations.
arXiv AI
Jun 30
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.
arXiv AI
Jul 7
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
arXiv:2603. 05659v3 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers.
arXiv AI
Jul 14
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
arXiv:2607. 10139v1 Announce Type: cross Abstract: Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution.
arXiv AI
Aug 28
From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoning
The paper investigates the reliability of rule- and model-based verifiers used in reinforcement learning with verifiable reward (RLVR) for mathematical reasoning. It finds that rule-based verifiers often miss equivalent answers in different formats, causing false negatives that degrade RL performance as models improve. Model-based verifiers achieve higher static accuracy but become vulnerable to reward hacking during RL, misclassifying certain response patterns as correct after fine-tuning.