arXiv:2607. 28591v1 Announce Type: cross Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation.
By Haomin Qi, Xingliang Wang, Xuanqi Gao, Baihui Sang, Xin Zhang, Minghua Ma, Pengfei Gao, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification.
Long-horizon coding requires agents to maintain an evolving view of the project they are changing: what problem is currently being solved, what has already been learned or modified, which other artifa...
SWE‑Duel proposes a self‑scaling benchmark for coding agents, where tasks and oracles are generated at evaluation time to prevent data leakage and adapt difficulty to participant performance. In the arena, two agents take turns as Red and Blue: Red proposes a feature, injects a subtle bug, and must repair it blind, while Blue reviews the pull request without knowing the bug and must pass all tests. A tournament of 11 model‑harness participants on 15 repositories across five languages showed GPT‑5.5/Codex CLI as the top solver and Claude Opus 4.8/mini‑swe‑agent as the top task author, with newer model versions producing harder tasks for older ones.
By Wei Minn, Zhensu Sun, Lwin Khin Shar, David Lo
arXiv:2610.09633v1 Announce Type: new
Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engine...
By Igor Slinko, Yaroslav Golubev, Sergey Titov
Coding agents can adapt their harnesses, the software, configuration, and instructions used to run them, based on experience from earlier tasks, but whether these adaptations improve their behavior on...