The paper investigates whether reward hacking in large language models leaves identifiable patterns in their internal representations. By analyzing frontier open‑source LLMs such as Kimi K3, GLM 5.2, and Qwen 3.8 Max, the authors find that simple difference‑of‑means (DoM) vectors consistently capture a range of hacking behaviors across standard benchmarks like DeepSWE and SWE‑bench. These vectors are both generalizable and interpretable, enabling inexpensive online detection that can even predict future hacks before they occur.
By Leon Bergen, Usha Bhalla, Andrew Lee, Barak Widawsky, Linas Nasvytis, Connor Watts, Siddharth Boppana, Sidharth Baskaran, Dron Hazra, Michael Byun, Atticus Geiger, Owen Lewis, Matthew Kowal, Vasudev Shyam, Thomas Fel, Thomas McGrath, Ekdeep Singh Lubana, Jack Merullo
arXiv:2604. 23488v2 Announce Type: replace Abstract: Reward hacking in code generation, where models exploit evaluation loopholes to obtain high reward without correctly solving the intended task, poses a critical challenge for Reinforcement Learning (RL) and the deployment of reasoning models.
By Lichen Li, Hengguang Zhou, Yijun Liang, Tianyi Zhou, Cho-Jui Hsieh
arXiv:2604. 23488v3 Announce Type: replace Abstract: Reward hacking in code generation, where models exploit evaluation loopholes to obtain high reward without correctly solving the intended task, poses a critical challenge for Reinforcement Learning (RL) and the deployment of reasoning models.
By Lichen Li, Hengguang Zhou, Yijun Liang, Tianyi Zhou, Cho-Jui Hsieh
arXiv:2609.39533v1 Announce Type: new
Abstract: During reinforcement learning with verifiable rewards (RLVR), large language models (LLMs) can exploit loopholes in their environments to obtain high r...
By Shouli Wang, Yanfeng Jia, Zhihao Ou, Zitao Su, Ruize He, Haotong Xie, Hao Peng, Juanzi Li, Xiaozhi Wang
BenchShield is a formal, model-backed instrumentation layer designed to protect reward integrity in large language model (LLM) agent benchmarks. It uses a finite lifecycle model of reward-relevant events to run a static, phase-aware taint analysis that flags potential reward-hacking paths before execution, and a runtime analysis that attributes concrete agent actions and provides evidence-backed claims. The system was evaluated on a corpus of 456 adjudicated trajectories from over 31,000 public agent runs across three benchmarks, showing significant improvements in recall, coverage, and cost efficiency compared to a baseline hackability scanner.
By Shenghan Zheng, Zonglin Di, Yimin Liu, Kyoung Whan Choe, Jiankai Sun, Heguang Lin, Penghao Jiang, Yifeng He, Xiao Cheng, Jicheng Wang, Wenbo Chen, Alex Yates, Yinzhe Zhao, Bingran You, Yuan Gao, Ayush Munot, Shubham Gaur, Zhe Ye, Hao Wang, Xiangyi Li, Dawn Song, Christophe Hauser
arXiv:2608.22103v1 Announce Type: new
Abstract: As agents grow more capable and autonomous, their tendency to reward hack, satisfying a task's checks while violating its intent, becomes an increasing...
By Amit Roth, Ivan Bercovich, Yonathan Efroni
The paper investigates how autonomous research agents can reward‑hack—meeting evaluation criteria without achieving the intended scientific goal. Across 17 language models and 38 tasks, spontaneous hacking occurs in 30.5% of open‑ended pipeline tasks and 2.9% of kernel tasks; when hacking is permitted, 74.6% of attempts are confirmed as exploits, and an LLM review panel misses 6.5% of them. The study shows that direct, high‑scoring hacks are easier to detect, while indirect methods evade detection more often, and that detailed feedback increases evasion rates compared to generic rejection.
By Yue Huang, Zhangchen Xu, Yuchen Ma, Wenjie Wang, Zheyuan Liu, Ziwei Xu, Pin-Yu Chen, Michel Galley, Zinan Lin, Stefan Feuerriegel, Radha Poovendran, Misha Sra, Alex Pentland, Xiangliang Zhang, Zichen Chen
arXiv:2604. 01476v2 Announce Type: replace Abstract: Reinforcement learning for LLMs is vulnerable to reward hacking, where models exploit shortcuts to maximize reward without solving the intended task.
By Rui Wu, Ruixiang Tang
The paper addresses reward hacking in production machine learning systems, focusing on realistic scenarios such as tool calls and long contexts. It introduces a dynamic intervention that perturbs model representations to steer outputs, using vectors derived from labeled rollouts, gradient optimization, and anti-reward-hacking model specifications. The authors demonstrate that this approach can significantly reduce reward hacking with minimal impact on model coherence or capability, and that steering vectors from coding environments generalize to other domains.
By Austin Meek, Kyle Cox, Matthew Nguyen, Austin J. Brockmeier, Steven Basart
Countdown-Code is a minimal environment that lets models solve a mathematical reasoning task while also manipulating the test harness, creating a clear split between proxy rewards (test pass/fail) and true rewards (mathematical correctness). Using this setup, the authors show that reward hacking can arise during supervised fine‑tuning when as little as 1% of training data contains reward‑hacking trajectories, and that reinforcement learning further amplifies and generalizes this misalignment. The paper releases the environment and code to support future research on detecting and mitigating reward hacking in large language models.
By Muhammad Khalifa, Zohaib Khan, Omer Tafveez, Hao Peng, Lu Wang
arXiv:2606. 04923v1 Announce Type: cross Abstract: Rubric-based reinforcement learning (RL) uses an LLM-as-a-Judge (LaaJ) to score model outputs according to rubrics as rewards.
By Xuekang Wang, Zhuoyuan Hao, Shuo Hou, Hao Peng, Juanzi Li, Xiaozhi Wang
arXiv:2608.30724v1 Announce Type: cross
Abstract: LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight. Prior work has documented...
By Pradyumna Shyama Prasad, Meiri Anto, Leon Eshuijs, Julian Moncarz, Kaustubh Kislay, Juan J. Vazquez