arXiv Machine Learning September 14, 2026 By Taoran Liang, Yang Liu, Shang Luo, Yingguang Yang, Rongrong Zhang, Yingzong Min, Yulin Huang, Jianshen Zhang, Yongzhi Qi, Kefu Xu, Congjing Ran, Bin Chong

Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces GACA, a critic‑free reinforcement learning estimator that adapts credit assignment granularity based on a step‑level uncertainty proxy. GACA assigns higher weight to fine‑grained signals for steps with above‑average negative log‑likelihood, while relying on episode‑level signals for less uncertain steps, improving task success on ALFWorld and WebShop for 1.5B and 7B language models. The authors provide a risk decomposition, a conditional bound on action‑value variation, and an error‑projection analysis to justify the method’s effectiveness.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 10

Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agents

arXiv:2608. 06861v1 Announce Type: new Abstract: Training large language model agents in long-horizon environments requires assigning credit from sparse terminal outcomes to individual actions.

By Hongxi Yan, Ziyue Huang, Shichao Fan, Qingjie Liu

llms agents multimodal

arXiv Machine Learning

Jun 5

When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training

arXiv:2606. 05885v1 Announce Type: new Abstract: Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards.

By Yuanfan Li, Qi Zhou, Wenjing Duan, Lu Chen

llms agents reinforcement-learning safety

arXiv AI

Jun 10

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

arXiv:2606. 09961v1 Announce Type: cross Abstract: Training large language models (LLMs) as autonomous agents via reinforcement learning (RL) has enabled frontier models to achieve superhuman performance in long-horizon tasks.

By Yu Han, Kailing Li, Yang Jiao, Yulin Dai, Yuqian Fu, Linhai Zhuo, Tianwen Qian

llms agents reinforcement-learning

arXiv AI

Aug 21

SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning

arXiv:2608. 19842v1 Announce Type: new Abstract: Agentic reinforcement learning (RL) has become a critical stage in the post-training of large language models.

By Dayang Liang, Lang Feng, Bo An, Yunlong Liu

llms agents reinforcement-learning

arXiv AI

Jun 30

BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards

arXiv:2606. 28707v1 Announce Type: new Abstract: Critic-free reinforcement learning with verifiable rewards (RLVR), exemplified by Group Relative Policy Optimization (GRPO), avoids training a value function (critic) and reduces memory and compute overhead relative to critic-based PPO pipelines for aligning large language models.

By Yupeng Chang, Yuan Wu, Yi Chang

llms reinforcement-learning benchmarks

Hugging Face Trending Papers

Aug 17

TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents

Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult. Existing approache...

llms agents reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agents

When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning

BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards

TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents