arXiv Machine Learning

Complementary RL: Towards Efficient Experience-Driven Agent Learning

arXiv:2603. 17621v2 Announce Type: replace Abstract: Reinforcement Learning (RL) has emerged as a powerful paradigm for training LLM-based agents, yet remains limited by low sample efficiency, stemming not only from sparse outcome feedback but also from the agent's inability to leverage prior experience across episodes.

arXiv AI
Aug 10

Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control

arXiv:2608. 07086v1 Announce Type: cross Abstract: Reinforcement learning systems are significantly more complex than other machine learning paradigms due to inherent properties, causing RL system design to jointly account for many tightly coupled factors.

By Qi Zhao, Guozheng Ma, Yilun Kong, Lu Li, Haoyu Wang, Zilin Wang, Tiantian Zhang, Yuxing Wang, Jian Sha, Yongzhe Chang, Xueqian Wang, Dacheng Tao
arXiv Machine Learning
Aug 28

Learning Generalizable Behaviors for Terminal Agents

The paper introduces the Agentic Compositional Generalization hypothesis, suggesting that reinforcement learning (RL) primarily refines high‑level decision‑making behaviors that orchestrate pre‑trained low‑level skills, rather than teaching new domain‑specific skills from scratch. It proposes River, a training recipe that enhances reward quality by filtering low‑quality synthetic environments and adding process‑level behavior regularization. Using River, RL‑trained agents outperform other open‑source 8B models on four terminal‑agent benchmarks, achieving significant gains with fewer than 30% of the training environments.

By Yihang Yao, Bo Pang, Xuan Phi Nguyen, Ding Zhao, Shafiq Joty, Semih Yavuz
arXiv AI
Jun 2

ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL

arXiv:2606. 01619v1 Announce Type: new Abstract: Agentic reinforcement learning (RL) enables LLM agents to improve continuously from environment rewards, yet the resulting policies do not systematically accumulate reusable strategies that generalize across tasks.

By Zelin He, Haotian Lin, Boran Han, Wei Zhu, Haoyang Fang, Bernie Wang, Xuan Zhu, Runze Li, Matthew Reimherr
arXiv AI
Sep 10

Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks

The paper introduces Feedback‑Enriched Environments (FEEs) as a new approach to training large language models as autonomous agents for long‑horizon tasks. By shifting from action guidance to observation enrichment during later stages of exploration, FEEs improve performance across SciWorld and BFCL benchmarks with various Qwen3 model scales and RL algorithms. The study shows that FEEs stabilize training, promote proactive exploration, embed environmental guidance into policy weights, and highlight intra‑group feedback consistency as key for stable optimization.

By Hongbang Yuan, Zhuoran Jin, Yixin Cao