RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.
By Binghang Lu, Yige Sun, Guang Lin
arXiv:2609.33180v2 Announce Type: replace-cross
Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies...
By Xiaojing Sun, Yuhan Zeng, Zihua She, Xiao Wang
RSI Arena is a two‑stage experiment where ten large language models, each starting from NVIDIA Nemotron 3.5 Lightning 30B-A3B Base, autonomously train and refine themselves using human feedback. In the first six‑day stage, each agent receives 1,000 GPU‑hours and $300 in API credit to select data, design training procedures, and choose checkpoints, producing working assistants that close a median 76% of the base‑to‑Instruct gap on 12 benchmarks. The second stage gives agents access to comparison records and an additional three days of training, resulting in style‑adjusted relative rating improvements for eight of the ten models, though benchmark gains remain modest and some agents exhibit longer responses or increased harmful compliance.
By Yue Huang, Yanbo Wang, Lichi Li, Yuanyuan Chen, Andreas Haupt, Xiaonan Luo, Zheyuan Liu, Jihye Choi, Felix Juefei-Xu, Xiangliang Zhang, Radha Poovendran, Misha Sra, Ying Liu, Ben Burtenshaw, Thomas Wolf, Alex Pentland, Zichen Chen, Zhangchen Xu
arXiv:2610.14625v1 Announce Type: new
Abstract: Recursive self-improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow language agents...
By Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang, Xuejun Wu, Xi Chen, Buyue Qian, Yaowei Zheng, Junhao Hu
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
By Qiankai Xu
arXiv:2610.02826v1 Announce Type: new
Abstract: Successful trajectories on difficult tasks provide valuable supervision for model improvement, but specialized harnesses introduce interventions that m...
By Zongxia Li, Yucheng Shi, Zhongzhi Li, Junyao Yang, Ruhan Wang, Chengsong Huang, Fuxiao Liu, Haitao Mi, Jordan Boyd-Graber, LeoweiLiang
arXiv:2610.20474v1 Announce Type: new
Abstract: Pretrained vision-language-action models (VLAs) often fail midway through unseen long-horizon tasks. Recursive self-improvement (RSI) offers a remedy,...
By Peidong Wang, Ying Chang, Zhiming Ma, Xiaocui Yang, Shi Feng, Daqi Liu, Yuzhe Yang, Xu Cao, Chuang Wang, Jian Su, Daling Wang
The paper introduces Recursive Reward Improvement (RRI), a method that enhances recursive self‑improvement loops by using an adversary to expose and fix weaknesses in model‑generated rubrics for open‑ended tasks. In RRI, an adversary crafts a rubric‑gaming answer, an honest answer is generated without rubric exposure, and up to two repair steps adjust the rubric so the honest answer scores higher. Experiments on HealthBench Professional show that RRI boosts adjusted scores by up to 12.4 points on a 27B model and yields significant gains on difficult and writing tasks, while also improving performance on unseen benchmarks PRBench Hard and ProfBench.
By Wei Dai, Qianchu Liu, Guanghui Qin, Xing Han, Tristan Naumann, Paul Pu Liang, Sheng Zhang
arXiv:2610.14224v1 Announce Type: new
Abstract: LLM agents' capabilities depend not only on their underlying models but also on their execution harnesses. Recursive self-improvement (RSI) offers a pr...
By Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu, Bhavana Dalvi Mishra, Yifei Ming, Han Yu, Rujun Han, Chen-Yu Lee, Tomas Pfister
R$^2$ Flow is a recursive self‑improvement framework for large‑language‑model agents that alternates policy learning, independent verification, and versioned skill‑library updates on a shared‑state orchestration graph. By merging histories that differ only in the order of independent steps, it pools evidence across equivalent executions and uses flow‑share readouts, signed utility ranks, verifier evidence, and residual‑variance plateaus to decide when and which skills to edit. Experiments on question answering, mathematical reasoning, interactive decision making, and code generation show that R$^2$ Flow improves task accuracy and library‑edit precision compared to heuristic orchestration, reinforcement learning, and other skill‑evolution baselines, and it transfers across different executors.
By Mingda Zhang, Qiang Huang, Yanjin Li, Zijia Wang, Qika Lin, Xiaoying Tang, Tiesunlong Shen
The paper investigates how pretraining and midtraining enable reward-based learning by providing necessary information and computation. It analyzes sequential state computation and contextual memory, showing that task‑independent source observations resolve ambiguities in reward adaptation. Experiments on pretrained Qwen2.5 checkpoints across eight worlds demonstrate that correct source and first‑operation supervision significantly improve success rates, and that memory replay and independent confirmation further enhance performance.
By Chiwun Yang, Xiaoyu Li
arXiv:2610.17609v1 Announce Type: new
Abstract: Training-free self-improvement lets an agent use feedback to revise the response it is about to return and to update the harness state that shapes how...
By Tianyi Xu, Yaolun Zhang, Yiran Wu, Joe Nguyen, Shengyu Dai, Saba Moeinizade, Zhenwen Shao, Qingyun Wu, Huazheng Wang