arXiv Machine Learning

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works

arXiv:2607. 21273v1 Announce Type: new Abstract: Dense per-step supervision is an appealing remedy for sparse-reward, long-horizon LLM agents: reward the agent for predicting its next observation, and memory should follow.

arXiv Machine Learning

Aug 5

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

arXiv:2607. 21273v2 Announce Type: replace Abstract: Dense per-step supervision is the standard remedy for sparse-reward long-horizon LLM agents: reward the policy for predicting its next observation, which looks provably safe under potential-based shaping.

By Yu Wang

llms agents

arXiv Machine Learning

Jul 15

When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

arXiv:2607. 11953v1 Announce Type: new Abstract: Does a reinforcement-learning agent that earns high reward represent its task's latent state, or only a reward-correlated shortcut?

By Jim Allchin

agents reinforcement-learning

arXiv AI

Aug 3

PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization

arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.

By Wonjoong Kim, Yeonjun In, Sangwu Park, Dongha Lee, Chanyoung Park

llms agents reinforcement-learning

arXiv AI

Jul 8

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.

By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun

llms agents

arXiv Machine Learning

Aug 6

Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning

arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.

By Jai Malegaonkar, Rohan Patil, Henrik I. Christensen

agents reinforcement-learning efficiency

arXiv Machine Learning

Jul 10

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.

By Jan Malte T\"opperwien, Aditya Mohan, Marius Lindauer

reinforcement-learning robotics

arXiv AI

Jul 21

CIGPO: Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading LLM Agents

arXiv:2607. 16244v1 Announce Type: cross Abstract: Training multi-turn evidence-reading agents with outcome-only reinforcement learning is unstable because intermediate turns receive little direct credit.

By Hao Dou

llms agents reinforcement-learning

arXiv Machine Learning

Jun 18

Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards

arXiv:2606. 18963v1 Announce Type: new Abstract: We study online reward-punishment learning when the environment provides no scalar reward or evaluative label.

By Zirong Li

agents

arXiv AI

Jul 15

A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism

arXiv:2607. 12640v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards, and Group Relative Policy Optimization (GRPO) in particular, is now run routinely on a supervised checkpoint in the hope of producing a stronger agent.

By Chengguang Gan, Zhixi Cai, Yunhao Liang, Hanjun Wei, Shiwen Ni, Qinghao Zhang

llms rag agents reinforcement-learning multimodal

arXiv AI

Jun 3

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

arXiv:2606. 03238v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) makes large-scale post-training possible by replacing an underspecified human objective with learned and scalable proxies.

By Zelalem Abahana

llms reinforcement-learning safety

arXiv AI

Jun 30

SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution

arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.

By Aojie Yuan, Yi Nian, Haiyue Zhang, Zijian Su, Yue Zhao

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jul 28

Progress-conditioned Group Policy Optimization for Long-Horizon Agentic Tasks

arXiv:2607. 22724v1 Announce Type: cross Abstract: Group-based policy optimization has been increasingly used to train large language model (LLM) agents from sparse outcome rewards by comparing trajectories or steps within a group.

By Kaibing Yang, Guangfeng Cai, Shengtian Yang, Shuo He, Yu Li, Mengyi Liu, Pengwei Chen, Jun Xu, Lei Feng

llms agents benchmarks

Related stories