arXiv Machine Learning
Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 28
HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
HINT-SD introduces a targeted self‑distillation framework for long‑horizon language‑model agents that uses full‑trajectory hindsight to identify failure‑relevant actions and applies feedback‑conditioned distillation only to those action spans. This selective approach reduces the need for per‑turn feedback, improving training efficiency and effectiveness. Experiments on BFCL v3 and AppWorld demonstrate that HINT‑SD outperforms dense per‑turn feedback baselines by up to 13.60 percentage points on average while cutting training time per step by 2.26×.
arXiv Machine Learning
Jul 31
Group-Reflective Self-Distillation for Agentic Reinforcement Learning
arXiv:2607. 28076v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents.
Hugging Face Trending Papers
Jul 30
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Recent advances in post-training Large Language Models (LLMs) increasingly rely on Reinforcement Learning with Verifiable Rewards (RLVR) or On-Policy Self-Distillation (OPSD). While OPSD provides dense, logit-level supervision, it inherently suffers from exposure bias due to the privileged information of the self-teacher.
arXiv Machine Learning
Jul 31
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
arXiv:2607. 28026v1 Announce Type: new Abstract: Recent advances in post-training Large Language Models (LLMs) increasingly rely on Reinforcement Learning with Verifiable Rewards (RLVR) or On-Policy Self-Distillation (OPSD).
arXiv AI
Jul 31
ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Tasks
arXiv:2606. 27814v4 Announce Type: replace Abstract: Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement.
arXiv AI
Aug 20
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
The paper introduces Reverse‑Turn Policy Optimization (RTPO), a method that restructures multi‑turn agentic reinforcement learning rollouts into sparse reverse trees and updates policies in temporal reverse order. This approach addresses three key instability sources—context mismatch, weak turn‑level credit assignment, and asynchronous policy drift—by aligning each decision with its downstream continuation. Theoretical analysis shows RTPO eliminates context mismatch and drift, reduces credit bias, and converges to recursive optimality, while experiments demonstrate performance gains of 21.50% over trajectory‑level and 10.76% over turn‑level baselines on multi‑turn agentic RL benchmarks.