arXiv Machine Learning July 24, 2026 By Yu Wang

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works

Read the original on arXiv Machine Learning →

arXiv:2607. 21273v1 Announce Type: new Abstract: Dense per-step supervision is an appealing remedy for sparse-reward, long-horizon LLM agents: reward the agent for predicting its next observation, and memory should follow.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 5

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

arXiv:2607. 21273v2 Announce Type: replace Abstract: Dense per-step supervision is the standard remedy for sparse-reward long-horizon LLM agents: reward the policy for predicting its next observation, which looks provably safe under potential-based shaping.

By Yu Wang

llms agents

arXiv Machine Learning

Jul 15

When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

arXiv:2607. 11953v1 Announce Type: new Abstract: Does a reinforcement-learning agent that earns high reward represent its task's latent state, or only a reward-correlated shortcut?

By Jim Allchin

agents reinforcement-learning

arXiv AI

Aug 3

PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.

By Wonjoong Kim, Yeonjun In, Sangwu Park, Dongha Lee, Chanyoung Park

llms agents reinforcement-learning

arXiv AI

Jul 8

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.

By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun

llms agents

arXiv Machine Learning

Aug 6

Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning

arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.

By Jai Malegaonkar, Rohan Patil, Henrik I. Christensen

agents reinforcement-learning efficiency

arXiv Machine Learning

Jul 10

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.

By Jan Malte T\"opperwien, Aditya Mohan, Marius Lindauer

reinforcement-learning robotics

Related stories

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning

Beyond Success Rates: Trainability and Extractability for Offline GCRL