arXiv Computer Vision By Jingdong Zhang, Jionghao Wang, Zhengming Yu, Yancheng Cai, Wei Chow, Chris Choy, Zhiyang Dou, Hang Hua, Jiachen Li, Zhixuan Liu, Chen Wang, Songlin Wei, Siyuan Yang, Yu Yuan, Xiaohang Zhan, Yucheng Zhang, Shuhong Zheng, Xin Li, Lingjie Liu, Wojciech Matusik, Zhengzhong Tu, Wenping Wang

Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents

Read the original on arXiv Computer Vision →

Flex‑RSI is a new benchmark that evaluates how agents can recursively improve their own harness—prompts, tools, memory, and control loop—while solving vision and robotics tasks under a fixed budget. Each round alternates between an executor that refines the task solution and an improver that revises the harness, with only higher‑scoring solutions retained for the next iteration. The benchmark tests both closed‑source and open‑weight models across perception, reasoning, robot action, 3D reconstruction, visual creation, and infrastructure tasks, revealing that self‑improvement is most effective when agents build their own solutions, that simple success/failure feedback limits open‑weight models, and that harness editing itself contributes significantly to performance gains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Oct 1

Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.

By Qiankai Xu