arXiv Machine Learning July 24, 2026 By Yu Wang

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works

Read the original on arXiv Machine Learning →

arXiv:2607. 21273v1 Announce Type: new Abstract: Dense per-step supervision is an appealing remedy for sparse-reward, long-horizon LLM agents: reward the agent for predicting its next observation, and memory should follow.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 5

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

arXiv:2607. 21273v2 Announce Type: replace Abstract: Dense per-step supervision is the standard remedy for sparse-reward long-horizon LLM agents: reward the policy for predicting its next observation, which looks provably safe under potential-based shaping.

By Yu Wang

llms agents

arXiv AI

Sep 2

Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents

The paper proposes CANOPY, a minimalist reinforcement learning protocol that addresses two common pitfalls—signal starvation and policy drift—in outcome‑only RL for long‑horizon interactive tasks. By scaling same‑task exploration, keeping updates on‑policy, and anchoring updates with KL divergence, CANOPY enables a Qwen3‑14B agent to achieve top leaderboard results on the AppWorld coding benchmark without auxiliary supervision or elaborate scaffolding. The approach also improves performance on SWE‑bench for a Qwen3.5‑9B model.

By Liming Pu, Xiaoxia Li, Yifu Liu, Teng Cao, Bin Yang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jul 15

When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

arXiv:2607. 11953v1 Announce Type: new Abstract: Does a reinforcement-learning agent that earns high reward represent its task's latent state, or only a reward-correlated shortcut?

By Jim Allchin

agents reinforcement-learning

arXiv AI

Aug 3

PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.

By Wonjoong Kim, Yeonjun In, Sangwu Park, Dongha Lee, Chanyoung Park

llms agents reinforcement-learning

arXiv AI

Jul 8

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.

By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun

llms agents

arXiv Machine Learning

Aug 6

Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning

arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.

By Jai Malegaonkar, Rohan Patil, Henrik I. Christensen

agents reinforcement-learning efficiency

One email a morning, machine-written

Related stories

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents

When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning