arXiv Machine Learning
Rent-Seeking Language Models: Reward Models Set the Prices, Frequent Audits Curb the Rent
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 7
Gradient Regularization Mitigates Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards
arXiv:2602. 18037v2 Announce Type: replace-cross Abstract: Reinforcement Learning from Human Feedback (RLHF) or Verifiable Rewards (RLVR) are two key steps in the post-training of modern Language Models (LMs).
arXiv AI
Oct 15
Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR
The paper surveys 58 studies on verifier errors in reinforcement learning with verifiable rewards (RLVR), examining how such errors affect training and which mitigations are effective. It finds that the type of error—rather than its frequency—has a larger impact on learning dynamics, with random errors mainly slowing progress and false positives potentially causing learning plateaus or collapse. The authors compile findings into comparison tables, reconcile conflicting reports, and propose a reporting protocol to standardize future research.
arXiv AI
Aug 17
Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation
arXiv:2608. 13564v1 Announce Type: new Abstract: Evaluating language-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time.
arXiv Computation and Language
Oct 1
CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL
arXiv:2609.39533v1 Announce Type: new Abstract: During reinforcement learning with verifiable rewards (RLVR), large language models (LLMs) can exploit loopholes in their environments to obtain high r...
arXiv AI
Jul 7
Out-of-Distribution Generalization of Risk Aversion in Language Models
arXiv:2607. 02755v1 Announce Type: cross Abstract: Training AIs to be risk-averse in resources could offer a failsafe in the event that AIs turn out misaligned.
arXiv Machine Learning
Oct 16
Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?
The paper investigates whether the reported out‑of‑domain performance gains of Reinforcement Learning with Verifiable Rewards (RLVR) are due to improved answer quality or simply to changes in answer formatting. By comparing scores from standard extractors with a custom extractor that distinguishes answers inside a oxed{} notation, the authors find that a large portion (about 82%) of the apparent gains on 7B SimpleRL models vanish, indicating that the gains are largely due to formatting rather than better answers. In contrast, a 32B model trained with a different recipe shows minimal format‑related gains, suggesting that the effect depends on the training setup.