arXiv Computation and Language
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 1
BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
arXiv:2608.30724v1 Announce Type: cross Abstract: LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight. Prior work has documented...
arXiv AI
Sep 10
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
arXiv:2609.08149v1 Announce Type: new Abstract: SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging repository-level tasks. However, our analys...
arXiv AI
Jun 9
SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.
arXiv Machine Learning
Jun 25
Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Trainig-Time Reward Hacking in Code Generation
arXiv:2604. 23488v2 Announce Type: replace Abstract: Reward hacking in code generation, where models exploit evaluation loopholes to obtain high reward without correctly solving the intended task, poses a critical challenge for Reinforcement Learning (RL) and the deployment of reasoning models.
arXiv Machine Learning
Aug 4
Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Training-Time Reward Hacking in Code Generation
arXiv:2604. 23488v3 Announce Type: replace Abstract: Reward hacking in code generation, where models exploit evaluation loopholes to obtain high reward without correctly solving the intended task, poses a critical challenge for Reinforcement Learning (RL) and the deployment of reasoning models.
arXiv AI
Aug 25
Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks
arXiv:2608.22103v1 Announce Type: new Abstract: As agents grow more capable and autonomous, their tendency to reward hack, satisfying a task's checks while violating its intent, becomes an increasing...