The paper investigates how to improve environment‑agnostic coding agents by refining prompts and leveraging scaffolding. By modifying a single instruction to encourage reporting difficulties and allowing automatic continuation, the authors achieve higher mean scores on ARC‑AGI‑3 and AGI Maze, though performance on Terminal‑Bench remains mixed. A scripted controller that handles reset decisions recovers part of the specialized agent’s advantage, while additional tutorial experience can actually reduce performance, underscoring the importance of effectively using available information and tools.
The study investigates how individual components of a coding harness—planning, action space, and context management—affect autonomous coding agents’ performance. By fixing the execution loop and varying these components across 176 settings on SWE‑Bench Verified and Terminal‑Bench 2.1, the authors find that context management is most valuable when context windows are tight, staging rule‑based elision before LLM summarization yields the best efficiency, planning serves as an accuracy scaffold for weaker models and a cost saver for stronger ones, and predefined tools help models with limited bash skills while bash‑capable models benefit from a bash‑only interface. Trajectory‑level analysis shows that context management lengthens execution paths, planning alters where trajectories terminate, and the action space determines code granularity, offering a modular framework for future harness design.
By Run-Ze Fan, Zihao Zhang, Simin Ma, Yebowen Hu, Shouju Wang, Kaiqiang Song, Fei Liu, Hamed Zamani, Xiaoyang Wang
arXiv:2605. 05138v2 Announce Type: replace Abstract: We evaluate an initial coding-agent system for ARC-AGI-3 in which the agent maintains an executable Python world model, verifies it against previous observations, refactors it toward simpler abstractions as a practical proxy for an MDL-like simplicity bias, and plans through the model before acting.
By Sergey Rodionov
arXiv:2607. 08124v1 Announce Type: cross Abstract: The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures.
By Jun Nie, Yonggang Zhang, Jun Song, Qianshu Cai, Dahai Yu, Yike Guo, Xinmei Tian, Bo Han
The paper investigates whether large language model–based coding agents can automatically synthesize programs that solve generalized task and motion planning (TAMP) problems across diverse instances. Using Claude Code and Codex, the authors evaluate 980 generated programs on 100 held‑out environments from KinDER and PDDLStream, achieving mean success rates between 56 % and 95 %—higher than hand‑engineered planners and other baselines—while requiring an order of magnitude less computation per instance. The study demonstrates that coding agents can calibrate physical models, test edge cases, and refine strategies, suggesting they are a strong baseline for generalized TAMP.
By Matteo Merler, Bowen Li, Josh Roy, Yichao Liang, Qianwei Wang, Yixuan Huang, Tom Silver
arXiv:2607. 15439v1 Announce Type: new Abstract: Our previous ARC-AGI-3 agent bundled executable world modeling, scheduled simplification, and exact replay verification, leaving unclear which idea accounted for its performance.
By Sergey Rodionov