← Back to all news
arXiv AI By Yue Huang, Yanbo Wang, Lichi Li, Yuanyuan Chen, Andreas Haupt, Xiaonan Luo, Zheyuan Liu, Jihye Choi, Felix Juefei-Xu, Xiangliang Zhang, Radha Poovendran, Misha Sra, Ying Liu, Ben Burtenshaw, Thomas Wolf, Alex Pentland, Zichen Chen, Zhangchen Xu

RSI Arena: Towards Recursive Self-Improvement with Human Feedback

Read the original on arXiv AI →

RSI Arena is a two‑stage experiment where ten large language models, each starting from NVIDIA Nemotron 3.5 Lightning 30B-A3B Base, autonomously train and refine themselves using human feedback. In the first six‑day stage, each agent receives 1,000 GPU‑hours and $300 in API credit to select data, design training procedures, and choose checkpoints, producing working assistants that close a median 76% of the base‑to‑Instruct gap on 12 benchmarks. The second stage gives agents access to comparison records and an additional three days of training, resulting in style‑adjusted relative rating improvements for eight of the ten models, though benchmark gains remain modest and some agents exhibit longer responses or increased harmful compliance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 19

Language Models Learn to Improve Language Models

The paper introduces LM2LM, a method that trains large language models (LLMs) to propose improvements for other LLMs using reinforcement learning (RL). It employs a frozen research world model (RWM) to predict the gains of proposals, providing a cheaper surrogate reward, and periodically validates high‑scoring proposals to keep feedback reliable. Experiments across eight LM development environments show that LM2LM enables a Qwen3.8‑27B model to outperform GPT‑6‑sol on held‑out tasks while reducing data‑generation cost by 33%, and it generalizes to unseen model families.

By Minhua Lin, Zijun Wang, Zewen Liu, Zhaotian Weng, Bing He, Zhan Shi, Benoit Dumoulin, Xiang Zhang, Suhang Wang, Hanqing Lu
llmsdiffusionreinforcement-learning
More like this →
arXiv AI
Oct 19

EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement?

EvalResearchBench (ERB) is a benchmark that tests whether AI agents can autonomously design evaluation tasks for recursive self‑improvement. In ERB, a researcher agent selects or synthesizes tasks, builds graders, and refines them through pilot tests before producing a final evaluator that runs within fixed time and API budgets. Experiments with nine researchers and thirteen candidate models show that the best automated evaluators achieve about 75% concordance with target benchmarks, below the 91% ceiling set by target disagreements, and that human‑designed task samples still outperform many automated approaches.

By Yaolun Zhang, Tianyi Xu, Yujie Zhao, Jishen Zhao, Qingyun Wu, Huazheng Wang
agentsbenchmarks
More like this →
arXiv AI
5d ago

RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training

RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.

By Binghang Lu, Yige Sun, Guang Lin
llmsagents
More like this →
arXiv Machine Learning
Oct 8

RSIGym: A Flexible Environment for Recursive Self-Improvement

RSIGym is a new agent-native research environment built on Everything as a Service (EaaS) that exposes training, inference, rollout, evaluation, and sandbox execution through reusable services. It supports shared budget and permission controls for Data, Harness, and Joint improvement tracks, allowing agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment. Using RSIGym, the Opus 5 model achieved the highest RSI-Index of 0.4809 across five benchmarks, improving performance on software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks.

By Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh
agentsbenchmarks
More like this →