arXiv Machine Learning

CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning

arXiv:2601. 15141v2 Announce Type: replace Abstract: Agentic Reinforcement Learning (RL) has empowered Large Language Models (LLMs) to utilize tools like Python interpreters for complex problem-solving.

arXiv AI
Sep 1

AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing

AgenticRag‑R1 is a reinforcement‑learning framework that integrates reasoning, retrieval, and memory through a stack and fine‑grained action space. It uses hierarchical action‑aware rewards and an information‑aware trajectory rejection strategy to support long‑horizon learning. Experiments on multi‑hop, open‑domain, and agentic reasoning benchmarks show that AgenticRag‑R1 outperforms strong baselines and produces robust, interpretable, memory‑aware reasoning behaviors.

By Xinke Jiang, Yue Fang, Zhibang Yang, Jiaran Gao, Zhixin Zhang, Tao Feng, Rihong Qiu, Wentao Zhang, Hongxin Ding, Ruizhe Zhang, Yongxin Xu, Yuheng Huang, Xu Chu, Junfeng Zhao, Yasha Wang
arXiv Machine Learning
Jul 16

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

arXiv:2607. 13988v1 Announce Type: new Abstract: Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training.

By Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge, Hao Cheng, Mike Hang Wang, Jianfeng Gao, Sharon Li
Hugging Face Trending Papers
Jun 17

Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation

Self-distillation improves reasoning in large language models by using the model's own rollouts as training signal, typically through implicit logit-level alignment that minimizes KL divergence toward a privileged target distribution. However, because this supervision is generated via uncontrolled sampling, it provides no diagnostic insight into the model's specific errors or corrective guidance for its individual failure patterns.

arXiv AI
Jul 23

In-the-Flow Agentic System Optimization for Effective Planning and Tool Use

arXiv:2510. 05592v2 Announce Type: replace Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios.

By Zhuofeng Li, Haoxiang Zhang, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou, Pan Lu
Hugging Face Trending Papers
Jul 21

REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

Large-scale online reinforcement learning (RL) is the predominant means of eliciting advanced abilities including long-term reasoning and agentic tool use in large language models (LLMs). However, continuing to scale it across vast task domains of interest remains challenging in both computational infrastructure and cost, especially when considering RL as merely a one-off learning stage.

arXiv AI
Sep 2

HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution

HarnessEvolve is a self‑evolving framework that improves agent harnesses—prompts, skills, tools, and execution logic—by learning from reference trajectories. It separates execution, evaluation, optimization, and gating into independent modules, addressing credit assignment failure, shortcut learning, and catastrophic forgetting. The approach uses reference trajectories to extract error signals, applies quality and performance gates to candidate updates, and validates updates on held‑out data, consistently outperforming state‑of‑the‑art baselines across diverse benchmarks.

By Wen Jiang, Mingmin Chu, Yimeng Tian, Qianxin Zhang, Haofei Yang, Rui Yang, Yang Liu, Tao Lv, Fangming Li