arXiv Machine Learning August 26, 2026 By Bo Ren, Yirong Mao, Yi Yang, Wenhui Que

IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents

Read the original on arXiv Machine Learning →

The paper introduces Influence-Aware Policy Optimization (IAPO), a method that models multi‑turn agent rollouts as typed influence‑dependency graphs to better assign credit to actions based on how information and errors flow through user and tool interactions. IAPO transforms the structure of support and failure usage into routing weights that redistribute trajectory‑level advantage, enabling more effective learning from sparse final rewards. Experiments with Qwen3‑4B and Qwen3‑8B on three service‑agent benchmarks show that IAPO outperforms existing multi‑turn reinforcement learning baselines without harming function‑calling performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Aug 25

IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents

Large Language Model (LLM) agents increasingly solve long-horizon tasks through multi-turn interactions with users and external tools. In these settings, relevant task information often unfolds over t...

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 16

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

arXiv:2607. 13988v1 Announce Type: new Abstract: Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training.

By Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge, Hao Cheng, Mike Hang Wang, Jianfeng Gao, Sharon Li

agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 11

APPO: Agentic Procedural Policy Optimization

arXiv:2606. 12384v1 Announce Type: cross Abstract: Recent advances in agentic Reinforcement Learning (RL) have substantially improved the multi-turn tool-use capabilities of large language model agents.

By Xucong Wang, Ziyu Ma, Yong Wang, Yuxiang Ji, Shidong Yang, Guanhua Chen, Pengkun Wang, Xiangxiang Chu

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 10

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

arXiv:2606. 09961v1 Announce Type: cross Abstract: Training large language models (LLMs) as autonomous agents via reinforcement learning (RL) has enabled frontier models to achieve superhuman performance in long-horizon tasks.

By Yu Han, Kailing Li, Yang Jiao, Yulin Dai, Yuqian Fu, Linhai Zhuo, Tianwen Qian

llms agents reinforcement-learning

arXiv AI

Jun 30

Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization

arXiv:2602. 11351v2 Announce Type: replace Abstract: Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns, enabling efficient task completion beyond passive instruction following and making them essential for real-world, user-centric applications.

By Yihang Yao, Zhepeng Cen, Haohong Lin, Shiqi Liu, Zuxin Liu, Jiacheng Zhu, Zhang-Wei Hong, Laixi Shi, Ding Zhao

llms agents reinforcement-learning benchmarks

arXiv AI

Sep 25

Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning

The paper introduces GRAFT, a Graph-based Faithful sTep-level credit-assignment framework that constructs a trajectory graph from rollout trajectories, recovers node state-values via Bellman iteration, and assigns step-level advantages based on node value differences. It also proposes Graph GAE to further reduce state-value estimation bias. Experiments on multi-turn agentic benchmarks demonstrate consistent improvements over GRPO and other recent agentic RL algorithms.

By Xincheng Yao, Haobo Fu, Weiming Liu, Chongyang Zhang

llms agents reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

APPO: Agentic Procedural Policy Optimization

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization

Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning