arXiv AI

Beyond Outcome Feedback: Trajectory-Quality-Guided Experience-based Learning

arXiv AI
Oct 16

MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling

MetaExp is a framework that trains language‑model agents to learn reusable experience across tasks by using verifiable cross‑task environment scaling. It introduces two designs—Shared‑Tool families that keep black‑box tools constant and Shared‑Workflow families that preserve latent procedures—allowing agents to acquire operational and procedural experience respectively. After fine‑tuning Qwen3.5‑122B‑A10B on thousands of trajectories from 1,000 task families, MetaExp achieves significant performance gains on benchmarks such as SpreadsheetBench, APEX‑Agents, and AutomationBench, with improvements persisting even when domain‑excluded training is applied.

By Renhao Li, Xinjie Shen, Yang Su, Jianhong Tu, Derek F. Wong, Dayiheng Liu, Min Yang
arXiv AI
Sep 15

CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection

arXiv:2604.07487v2 Announce Type: replace Abstract: Large language model agents rely on effective model context to obtain task-relevant information for decision-making. Many existing context engineer...

By Linbo Liu, Guande Wu, Han Ding, Yawei Wang, Qiang Zhou, Yuzhe Lu, Zhichao Xu, Huan Song, Panpan Xu, Lin Lee Cheong
arXiv Machine Learning
Oct 8

ExperienceIndex: Artifact-Grounded Memory

arXiv:2610.10091v1 Announce Type: cross Abstract: Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literatu...

By Peter Baile Chen, Geoffrey X. Yu, Xinming Liu, Samuel Madden, Dan Roth, Jacob Andreas, Doug Downey, Michael Cafarella
Hugging Face Trending Papers
Aug 3

CRISP: Critical Step Perception for Training Efficient Deep Search Agents

Large language models (LLMs) are increasingly extended into deep search agents that solve complex questions through multi-step interaction with external search and browsing tools. However, existing agents often incur substantial computational and interaction costs, generating lengthy trajectories that contain redundant queries, inefficient exploration, and irrelevant observations.

arXiv Machine Learning
5d ago

TalentRL: Strategy-Guided Reinforcement Learning for Hierarchical Skill Routing

TalentRL is a strategy‑guided reinforcement learning framework that routes natural‑language skills hierarchically, using coarse‑grained talents to guide fine‑grained skill retrieval. The method trains talent routing via reward contrasts and optimizes execution with group‑relative advantages, achieving significant performance gains on open‑domain QA benchmarks and agentic tasks such as ALFWorld and WebShop.

By Huancheng Chen, Xiaodi Sun, Zhaoqiong Huang, Michelle Chang, Jingwen Lu
arXiv AI
Oct 21

TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment

TrajEvolve introduces a trajectory evolution paradigm with hindsight credit assignment to improve reinforcement learning for large language models on long-horizon tasks. It refines low-quality steps in historical trajectories to expose the policy to high-quality compositions that are rarely sampled, and reformulates step-level continuous credit assignment as a binary classification problem to identify redundant steps. Experiments on two datasets show that this approach outperforms existing methods.

By Yi Wen, Hao Chen, Wanyu Wang, Maolin Wang, Pengyue Jia, Derong Xu, Huize Tan, Yingyi Zhang, Wenlin Zhang, Weihong Luo, Xiku Du, Xiangyu Zhao
arXiv AI
Aug 19

PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs

PlanPO introduces a group planning-aware policy optimization method for multi-turn agentic large language models, addressing the issue of advantage collapse caused by treating all successful trajectories equally. By incorporating coarse-to-fine advantage signals that reflect differences in trajectory and turn lengths, PlanPO encourages agents to learn generalizable planning and generation behaviors. Experiments show a 27.2% average improvement over GRPO on benchmarks such as ALFWorld, WebShop, and SciWorld, with minimal extra training cost.

By Dayang Liang, Liyuan He, Xuan Feng, Shuxin Li, Bo An, Yunlong Liu
arXiv AI
Oct 7

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?

Learn2Play Bench is a new benchmark that tests how well large language model agents learn from experience in unfamiliar, text‑based games with novel or counterintuitive rules. The benchmark provides reproducible feedback, automatic scoring, and varied game instances to evaluate learning across repeated attempts and transfer to new situations. Findings show that retaining full action records aids learning, human players outperform agents, and the choice of harness significantly impacts performance and inference cost.

By Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi