arXiv Machine Learning
One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 30
Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
arXiv:2609.36178v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents. However, its uniform assignment o...
arXiv AI
3d ago
Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
Group-Graph Policy Optimization (G2PO) is a novel group-based reinforcement learning algorithm designed for long-horizon agentic tasks. It transforms linear interaction trajectories into a global state‑transition graph, aggregates identical observations across trajectories to reduce sampling variance, and redefines actions as transitions between state nodes. By employing an edge‑centric advantage estimation and globally standardizing TD errors, G2PO identifies and prioritizes critical transitions, achieving up to 22.2% higher success rates than state‑of‑the‑art baselines on benchmarks such as WebShop, ALFWorld, and AppWorld.
arXiv Machine Learning
Oct 15
Contextual Credit Policy Optimization
arXiv:2610.19103v1 Announce Type: new Abstract: Group-based reinforcement learning enables large language model agents to learn from sparse outcome rewards in long-horizon tasks. Recent methods deriv...
Hugging Face Trending Papers
Oct 13
ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning
Reinforcement learning (RL) has emerged as the primary paradigm for optimizing Large Language Model (LLM) agents, yet efficient critic-free algorithms (e.g., GRPO) suffer from a fundamental uniform cr...
arXiv AI
Oct 15
ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning
The paper introduces ProAS, a Progress‑Aware Advantage Shaping framework designed to improve critic‑free reinforcement learning for large language model agents. ProAS uses a dynamic Agentic Reference Graph to provide fine‑grained credit assignment through Masking Unreasonable Incentives and Asymmetric Penalty Redistribution, avoiding the uniform advantage broadcast that can reinforce invalid actions. Experiments on Qwen2.5‑1.5B‑Instruct and Qwen2.5‑7B‑Instruct show that ProAS raises baseline success rates to 91.9% and 94.3% on ALFWorld and 66.7% and 70.8% on WebShop, while speeding up training convergence up to 2.44× and remaining robust to imperfect reference graphs.
Hugging Face Trending Papers
Aug 25
IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents
Large Language Model (LLM) agents increasingly solve long-horizon tasks through multi-turn interactions with users and external tools. In these settings, relevant task information often unfolds over t...