arXiv Computation and Language

Evolving with Open Worlds: Towards Recursive Self-Improvement for Future Forecasting

arXiv AI
Sep 25

Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement

The paper introduces Env‑Rethink, a 27B post‑trained model system designed to help large language model agents better interact with complex, evolving environments. It builds Collection Maps and Event Logs to organize scattered information, uses offline trajectory learning to detect noise, and generates virtual event histories to evolve environments for more challenging tasks. Experiments show that Env‑Rethink improves downstream task performance by over 15.1% rubric pass rate across nine models on 30 tasks.

By Yukai Wu, Yuanjing Yang, Le Zhou, Shaokun Han, Haoyu Wang, Zirui Tang, Weihuang Zheng, Maxm Pan, Xuanhe Zhou, Fan Wu
arXiv AI
Oct 12

EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

EvoBelief is a self‑evolving world model that represents environment knowledge as falsifiable hypotheses, allowing an LLM agent to update its beliefs during deployment. The system generates hypotheses when unexpected outcomes or unknowns arise, validates them through observations and controlled experiments, and then uses the confirmed rules to guide predictions or plans. Across benchmarks such as ALFWorld, MiniGrid, and DiscoveryWorld, EvoBelief improves task performance, revises stale rules, and discovers quantitative laws, outperforming baseline frozen world models.

By Janet Wang, Jiang Guo, Yi Xiang, Jiaqing Yuan, Mingwen Dong, Hideo Kobayashi, Shuaichen Chang, Chandan K. Reddy, Jiarong Jiang
arXiv AI
Jun 26

The Red Queen G\"odel Machine: Co-Evolving Agents and Their Evaluators

arXiv:2606. 26294v1 Announce Type: cross Abstract: Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains.

By Alex Iacob, Andrej Jovanovi\'c, William F. Shen, Daniel Burkhardt, Meghdad Kurmanji, Nurbek Tastan, Lorenzo Sani, Niccol\`o Alberto Elia Venanzi, Ambroise Odonnat, Zeyu Cao, Bill Marino, Xinchi Qiu, Nicholas D. Lane
arXiv AI
Aug 20

SPADE: Self-Play in Adaptive Synthetic Executable Environments

SPADE (Self-Play in Adaptive Synthetic Executable Environments) is a reinforcement‑learning framework where a single large language model acts as both an Environment Designer—creating executable, long‑horizon training environments—and a Reasoning Agent—learning to act within those environments. The framework uses a regret signal based on the difference between rewarded performance with and without privileged hints to guide the Designer toward environments that are challenging yet solvable. Experiments show that, when scaled to 30‑billion‑parameter models, SPADE outperforms fixed‑environment baselines by significant margins across math, science, code, and reasoning benchmarks, and improves tool‑use performance on BFCL‑v4 and ACEBench‑Agent. whyItMatters":"By making environment design a learnable component, SPADE enables continuous self‑improvement and demonstrates that adaptive, self‑generated training environments can substantially boost language‑model performance across diverse tasks."

By Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques
arXiv AI
Oct 19

Darwin-Claw: Environment-Adaptive Experience Consolidation for Recursively Self-Evolving Agents

Darwin‑Claw is a new framework for large language model agents that enables environment‑adaptive, self‑evolving behavior. It systematically distills episodic experiences into prototype vectors through a hierarchical experience consolidation system, providing context‑sensitive perception patches and explicit optimization signals for skill refinement. The method is deployable with mainstream agent systems and, according to experiments on WildClawBench, delivers significant performance gains while maintaining controllability and stability.

By Mingze Yin, Xiaohan Wang, Dian Li, Longfei Luo, Jiahong Yan, Yiheng Zhu, Chang-Yu Hsieh
arXiv AI
Oct 15

From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement

arXiv:2610.14625v1 Announce Type: new Abstract: Recursive self-improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow language agents...

By Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang, Xuejun Wu, Xi Chen, Buyue Qian, Yaowei Zheng, Junhao Hu
arXiv Machine Learning
Sep 23

Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning

Agent0 is a fully autonomous framework that enables large language model agents to evolve without external data by using a multi‑step co‑evolution process. It pits a curriculum agent against an executor agent, both derived from the same base LLM, where the curriculum agent creates increasingly challenging tasks and the executor learns to solve them. By integrating external tools into the executor’s workflow, the system creates a self‑reinforcing cycle that continuously generates high‑quality curricula, leading to significant gains in reasoning performance—an 18% improvement on mathematical reasoning and 24% on general reasoning for the Qwen3‑8B‑Base model.

By Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, Yiyang Zhou, Caiming Xiong, Huaxiu Yao
arXiv AI
Sep 4

CoMAP: Co-Evolving World Models and Agent Policies for LLM Agents

CoMAP introduces a framework that jointly evolves textual world models and agent policies through a closed‑loop interaction. At each decision step the world model forecasts future state feedback for candidate actions, while the agent reflects on the reliability of this feedback to refine its action. The resulting on‑policy trajectories are used to self‑distill and update the world model, improving prediction accuracy and long‑horizon decision‑making across embodied planning, web navigation, and tool‑use benchmarks.

By Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li