arXiv Machine Learning
Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
5d ago
Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following
Verifiable instruction-following benchmarks emphasize output-constraint satisfaction but can overlook whether responses meaningfully answer the underlying request, creating an evaluation blind spot. W...
arXiv Computation and Language
Sep 1
Small Reward Models via Backward Inference
arXiv:2602.13551v3 Announce Type: replace Abstract: Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant...
arXiv AI
Jul 7
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
arXiv:2607. 04412v1 Announce Type: new Abstract: Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals.
arXiv Computation and Language
Aug 25
Detecting and Suppressing Reward Hacking with Gradient Fingerprints
The paper introduces Gradient Fingerprint (GRIFT), a technique that uses a model’s internal gradient computations to detect reward hacking in reinforcement learning with verifiable rewards. GRIFT compresses gradients of a chain-of-thought (CoT) conditioned on a prompt into a compact representation, which is then used to assess whether the CoT reflects reward hacking. Experiments on math, code, and logical reasoning benchmarks show GRIFT outperforms baselines by over 25% and, when integrated into a rejection fine‑tuning pipeline, reduces reward hacking while improving task performance.
arXiv Machine Learning
Jul 7
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
arXiv:2607. 04332v1 Announce Type: new Abstract: In this paper, we consider the setting where large language models (LLMs) are trained using reinforcement learning (RL) to simultaneously improve reasoning accuracy and verbalize its confidence.
arXiv AI
Jul 2
Reasoning Up the Instruction Ladder for Controllable Language Models
arXiv:2511. 04694v5 Announce Type: replace-cross Abstract: As large language model (LLM) based systems take on high-stakes roles in real-world decision-making, they must reconcile competing instructions from multiple sources within a single prompt context.