arXiv Computation and Language

RSI-Anything: Unlocking Recursive Self-Improvement with World Models

RSI-Anything is a domain-general framework that leverages predictive world models to guide action and learning-data selection for recursive self-improvement. It uses observed feedback to improve policies or workflows and adapts world models conditionally, while component and compatibility checks control the retention of updated policy-model pairs. Across multiple domains—including code repair, GUI tasks, laboratory planning, reaction optimization, and immune‑science applications—RSI-Anything shows substantial performance gains, such as higher success rates, reduced actions, doubled planning scores, and superior yield compared to baseline methods.

arXiv AI
3d ago

RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training

RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.

By Binghang Lu, Yige Sun, Guang Lin
arXiv Machine Learning
Oct 8

RSIGym: A Flexible Environment for Recursive Self-Improvement

RSIGym is a new agent-native research environment built on Everything as a Service (EaaS) that exposes training, inference, rollout, evaluation, and sandbox execution through reusable services. It supports shared budget and permission controls for Data, Harness, and Joint improvement tracks, allowing agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment. Using RSIGym, the Opus 5 model achieved the highest RSI-Index of 0.4809 across five benchmarks, improving performance on software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks.

By Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh
arXiv Computation and Language
Oct 9

Language Models as AI Research World Models

The paper proposes using language models as Research World Models (RWMs) to predict outcomes of experimental interventions across diverse research environments. By training on over 2,600 experimental records and 171,000 GPU‑hours, RWMs improve prediction accuracy within and across environments, reducing selection regret by up to 78% and boosting gains in multi‑round autoresearch. Ablation studies show that incorporating research knowledge yields greater benefits than merely changing the language model or increasing reasoning effort.

By Zijun Wang, Zewen Liu, Minhua Lin, Zhaotian Weng, Zhan Shi, Bing He, Yisi Sang, Dakuo Wang, Benoit Dumoulin, Wei Jin, Yuyin Zhou, Cihang Xie, Hanqing Lu
arXiv AI
Oct 12

EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

EvoBelief is a self‑evolving world model that represents environment knowledge as falsifiable hypotheses, allowing an LLM agent to update its beliefs during deployment. The system generates hypotheses when unexpected outcomes or unknowns arise, validates them through observations and controlled experiments, and then uses the confirmed rules to guide predictions or plans. Across benchmarks such as ALFWorld, MiniGrid, and DiscoveryWorld, EvoBelief improves task performance, revises stale rules, and discovers quantitative laws, outperforming baseline frozen world models.

By Janet Wang, Jiang Guo, Yi Xiang, Jiaqing Yuan, Mingwen Dong, Hideo Kobayashi, Shuaichen Chang, Chandan K. Reddy, Jiarong Jiang
arXiv AI
Sep 24

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

The paper introduces the Agent-Editing World Model (AEWM), a new approach that models how reasoning and actions influence future task progress instead of simulating tool responses. AEWM includes an Action Judge that classifies decisions as Critical, Exploratory, or Noisy, and a State Revision mechanism that edits noisy reasoning–action continuations from the same observed history. The integrated system, EditAct, directly updates the underlying state during real execution, leading to significant performance gains across multiple benchmarks and agent backbones.

By Shuang Sun, Guoxin Chen, Fanzhe Meng, Jia Deng, Huatong Song, Jinhao Jiang, Wayne Xin Zhao, Hongteng Xu, Ji-Rong Wen
arXiv Computation and Language
Sep 15

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

arXiv:2609.14857v1 Announce Type: new Abstract: Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve executio...

By Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, Chenghua Lin
arXiv AI
6d ago

Self-Improving Agents with World Model Predictive Paths

The paper introduces a test‑time framework that blends Model Predictive Path Integral (MPPI) control with a dual‑branch experience memory for large language model agents. At each decision step, the agent generates multiple candidate actions, uses a world model to simulate their outcomes, and evaluates these predictions for task progress and risk to select the best action. The dual‑branch memory allows the planner to refine action generation and the world model to improve future observation predictions, leading to strong performance on ALFWorld and ScienceWorld benchmarks.

By Haori Lu, Tianyu Gao, Yaoyao Liu, Yingying Li