The paper introduces "environment evolution," a method that incrementally raises the difficulty of interactive environments off‑policy, scheduling their generation across training generations to supply continuous learning signals. It derives three evolution directions tied to a multi‑turn learning objective and implements them via a loop‑engineered multi‑agent harness. Experiments with models such as Hy4 preview, Claude Opus 5, GPT‑5.6 Sol, Qwen3.6‑27B, and Qwen3.6‑35B‑A3B demonstrate that this approach consistently creates harder environments and boosts terminal‑agent performance on Terminal‑Bench 2.1 by 14.4–18.0 percentage points.
By Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu, Yun Yang, Dingxin Hu, Zhuo Han, Xing Wu, Feng Zhang, Lilin Wang
SPADE (Self-Play in Adaptive Synthetic Executable Environments) is a reinforcement‑learning framework where a single large language model acts as both an Environment Designer—creating executable, long‑horizon training environments—and a Reasoning Agent—learning to act within those environments. The framework uses a regret signal based on the difference between rewarded performance with and without privileged hints to guide the Designer toward environments that are challenging yet solvable. Experiments show that, when scaled to 30‑billion‑parameter models, SPADE outperforms fixed‑environment baselines by significant margins across math, science, code, and reasoning benchmarks, and improves tool‑use performance on BFCL‑v4 and ACEBench‑Agent.
whyItMatters":"By making environment design a learnable component, SPADE enables continuous self‑improvement and demonstrates that adaptive, self‑generated training environments can substantially boost language‑model performance across diverse tasks."
By Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques
arXiv:2606. 03108v1 Announce Type: new Abstract: Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static.
By Guhong Chen, Yingcheng Shi, Yongbin Li, Binhua Li, Xander Xu, Hu Wei, Shiwen Ni, Min Yang, Jieping Ye
The paper introduces "environment evolution," a method that incrementally raises the difficulty of interactive environments for terminal agents using off‑policy updates. It derives three evolution directions tied to a multi‑turn learning objective and implements them via a loop‑engineered multi‑agent harness. Experiments with models such as Hy4 preview, Claude Opus 5, GPT‑5.6 Sol, Qwen3.6‑27B, and Qwen3.6‑35B‑A3B demonstrate that this approach consistently generates harder environments and yields significant performance gains on Terminal‑Bench 2.1.
RISED introduces a framework that uses rubric-based textual feedback to improve training of a single large language model (LLM) agent across multiple interactive environments. By having an LLM judge tag rollouts with a shared rubric vocabulary, the system guides both online data selection and policy supervision, enabling richer cross‑environment relationships and within‑group reward contrast. Experiments show that RISED achieves the highest mean pass rate and ranks first or second in every individual environment, with rubric analysis revealing behavioural changes behind these gains.
By Jingtan Wang, Sirajul Salekin, Young mok Jung, Javier Movellan, Bryan Kian Hsiang Low, Manjot Bilkhu
AgentFly is an agentic reinforcement learning framework that introduces a unified resource layer to manage heterogeneous environments—such as sandboxes, model services, and external APIs—through a single scheduling engine. The framework is organized into four layers: an agent layer that abstracts agents, tools, and rewards; a rollout layer that composes agent loops and computes rewards; a context layer that organizes rollouts and injects contextual information; and a low‑level resource layer that handles resource management. AgentFly offers prebuilt tools and environments, demonstrates successful agent training across multiple tasks and models, and provides the first controlled cross‑framework throughput comparison against other agentic RL frameworks.
By Renxi Wang, Rifo Ahmad Genadi, Bilal El Bouardi, Yongxin Wang, Fajri Koto, Zhengzhong Liu, Timothy Baldwin, Haonan Li
arXiv:2606. 07412v1 Announce Type: cross Abstract: LLM-driven software engineering agents have become a central testbed for real-world language-model capability, yet their training remains limited by the availability of high-quality SWE tasks.
By Chuan Xiao, Zhengbo Jiao, Shaobo Wang, Wei Wang, Bing Zhao, Hu Wei, Linfeng Zhang, Lin Qu
HarnessEvolve is a self‑evolving framework that improves agent harnesses—prompts, skills, tools, and execution logic—by learning from reference trajectories. It separates execution, evaluation, optimization, and gating into independent modules, addressing credit assignment failure, shortcut learning, and catastrophic forgetting. The approach uses reference trajectories to extract error signals, applies quality and performance gates to candidate updates, and validates updates on held‑out data, consistently outperforming state‑of‑the‑art baselines across diverse benchmarks.
By Wen Jiang, Mingmin Chu, Yimeng Tian, Qianxin Zhang, Haofei Yang, Rui Yang, Yang Liu, Tao Lv, Fangming Li
arXiv:2608.05446v2 Announce Type: replace
Abstract: Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, recover from failures, and reuse experie...
By Xuying Ning, Dongqi Fu, Tianxin Wei, Yuanchen Bei, Xiyuan Yang, Wujiang Xu, Yueqi Song, Bingxuan Li, Zihao Li, Hanqing Zeng, Xiang Shen, Yajuan Wang, Yifan Wu, Qifan Wang, Jiayi Liu, Hong Li, Yinglong Xia, Xiangjun Fan, Hanghang Tong, Jingrui He
CoMAP introduces a framework that jointly evolves textual world models and agent policies through a closed‑loop interaction. At each decision step the world model forecasts future state feedback for candidate actions, while the agent reflects on the reliability of this feedback to refine its action. The resulting on‑policy trajectories are used to self‑distill and update the world model, improving prediction accuracy and long‑horizon decision‑making across embodied planning, web navigation, and tool‑use benchmarks.
By Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li
arXiv:2606. 02372v1 Announce Type: new Abstract: Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution.
By Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li
Agent Lightning v1.0 is a lightweight framework that enables harnessed agentic reinforcement learning, where the agent harness—managing tools, context, and control flow—directly participates in model post‑training. It supports arbitrary agent harnesses and addresses challenges such as retokenization, sample merging, and advantage calculation, providing a reproducible pipeline for instruction‑following, search, and coding agents. In experiments, RL training on 6K examples improved Qwen3.5‑9B’s performance on SWE‑bench from 41.8% to 56.4%.
By Zhiyuan He, Siwei Zhang, Zhiwen Zhou, Yuqing Yang, Yu Kang, Yuge Zhang, Luna K. Qiu, Tin Yan Tsui, Jiahang Xu, Chong Luo