← Back to all news
arXiv AI By Yingxuan Zhuang, Binhe Yu, Jingxiao Yang, Ruopei Sun, Ziting Li, Cheng Tan, Xuhong Zhang, Jianwei Yin, Jintao Chen

Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization

Read the original on arXiv AI →

The paper introduces BATON, a dual‑axis policy optimization framework for large language model agents that addresses two key aspects of reinforcement learning: intra‑trajectory feedback attribution and inter‑trajectory objective aggregation. BATON combines Bayesian Feedback Attribution, which builds a feedback‑conditioned posterior over sampled actions, with Trajectory Mass Normalization, which ensures equal optimization weight for complete trajectories. Experiments on ALFWorld, WebShop, and SearchQA demonstrate that each axis independently improves performance and that their combination yields the best results across different model scales.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 11

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.

By Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, Max Simchowitz
diffusionreinforcement-learningroboticsfine-tuningbenchmarks
More like this →
arXiv AI
Oct 19

Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning

Group-Graph Policy Optimization (G2PO) is a novel group-based reinforcement learning algorithm designed for long-horizon agentic tasks. It transforms linear interaction trajectories into a global state‑transition graph, aggregates identical observations across trajectories to reduce sampling variance, and redefines actions as transitions between state nodes. By employing an edge‑centric advantage estimation and globally standardizing TD errors, G2PO identifies and prioritizes critical transitions, achieving up to 22.2% higher success rates than state‑of‑the‑art baselines on benchmarks such as WebShop, ALFWorld, and AppWorld.

By Yunan Wang, Minghui Song, Zihan Zhang, Shaohan Huang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
llmsagentsreinforcement-learningefficiencybenchmarkssafety
More like this →
Hugging Face Trending Papers
Aug 20

RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation

Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction. Yet prevalent diffusion- and flow-matching robot policies lack tractable likelihoods, limiting their use in likelihood-based offline RL post-training.

diffusionreinforcement-learningroboticsefficiencybenchmarks
More like this →
arXiv Machine Learning
Jun 5

Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents

arXiv:2606. 05296v1 Announce Type: new Abstract: LLM agents operate in two distinct regimes: open-weight agents amenable to reinforcement learning (RL) and black-box agents whose behaviour must be controlled purely at test time.

By Dae Yon Hwang, Raunaq Suri, Valentin Villecroze, Anthony L. Caterini, Jesse C. Cresswell, No\"el Vouitsis, Brendan Leigh Ross
llmsagentsreinforcement-learningbenchmarks
More like this →