arXiv Machine Learning August 11, 2026 By Ibne Farabi Shihab, Fariya Afrin

Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify

Read the original on arXiv Machine Learning →

arXiv:2608. 08008v1 Announce Type: new Abstract: Process reward models (PRMs) score intermediate reasoning steps and are widely used for search, ranking, and training, but optimization can exploit these learned proxies by increasing reward while turning correct reasoning into incorrect reasoning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 21

Teach it to stop, not just to click

arXiv:2607. 17136v1 Announce Type: cross Abstract: Agentic computer-use RL is reported in single runs, and those numbers mislead.

By Barada Sahu (Cabal AI), Shivesh Pandey (Para AI)

agents reinforcement-learning efficiency multimodal

arXiv AI

Jun 16

Auditing Reward Hackability in Code RL Training Environments

arXiv:2606. 16062v1 Announce Type: new Abstract: We measure the rate at which code RL environments accept incorrect solutions as correct.

By Shreshth Rajan

llms reinforcement-learning

arXiv AI

Jun 30

Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

arXiv:2606. 30627v1 Announce Type: cross Abstract: Conservative offline training is widely advocated as a safe foundation for subsequent online adaptation: if a policy stays close to well-supported behaviour, the argument goes, it is less likely to exploit imperfections in a learned reward model.

By Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary

reinforcement-learning safety

arXiv Machine Learning

Jul 14

When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR

arXiv:2607. 11022v1 Announce Type: new Abstract: The test suites used as RLVR rewards for code have natural false positives: per-task, persistent, asymmetric errors that accept the same wrong programs every time they appear, unlike the symmetric or resampled noise assumed by existing noise-robustness analyses.

By Chuyifei Zhang

arXiv AI

Jun 3

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

arXiv:2606. 03238v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) makes large-scale post-training possible by replacing an underspecified human objective with learned and scalable proxies.

By Zelalem Abahana

llms reinforcement-learning safety

arXiv Machine Learning

Jun 16

Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Models

arXiv:2606. 16999v1 Announce Type: cross Abstract: Frozen small code models ( =45.

By Mehmet Iscan

benchmarks safety

Related stories

Teach it to stop, not just to click

Auditing Reward Hackability in Code RL Training Environments

Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Models