StateTape introduces a new framework for long‑horizon coding agents that rewrites the agent’s context as the code repository changes, rather than letting the context grow with every observation. It models the repository as a symbol‑level code graph, using a tape to mark symbols altered by each write and a manager model to resolve stale records. The authors provide theoretical analysis, a new benchmark called TraceBench, and empirical results showing higher resolve rates across six agents and three edit‑heavy benchmarks with minimal computational overhead.
By Ziyang Yu, Liang Zhao, Bowen Zhu, Hasibul Haque
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification.
arXiv:2607. 28591v1 Announce Type: cross Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation.
By Haomin Qi, Xingliang Wang, Xuanqi Gao, Baihui Sang, Xin Zhang, Minghua Ma, Pengfei Gao, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang
The paper introduces CTXGAP, a study measuring how much information coding agents omit from their user-facing reports compared to their full execution logs. Across 37 coding tasks, agents answer 93.6% of task-state questions when asked directly, but report-only readers recover only 52.0% of the same information, revealing a 40.7‑point gap. Even at final checkpoints of verified fixes, the gap remains significant, and detailed terminal scrollback improves recovery only to 73.6%. The study highlights that missing details in reports can leave supervisors unaware of failures, workarounds, and risks that need scrutiny.
By Guoyang Xu, Rui Zhang, Xiuyu Wu, Hao Chen
arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.
By Mohit Raghavendra, Anisha Gunjal, Aakash Sabharwal, Yunzhong He
arXiv:2610.14146v1 Announce Type: new
Abstract: Long-horizon terminal benchmarks measure whether an agent can sustain progress through extended workflows with process rewards, not only whether its fi...
By Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Chengsong Huang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ergan Shang, Fuxiao Liu, Tong Zheng, Ninghao Liu, Lichao Sun, Haitao Mi, Leowei Liang
StagedWorkspace is a versioned workspace designed for knowledge‑work AI agents, ensuring that every parsed view, native file edit, and review diff is explicitly tied to a specific version of the workspace state. By binding parsed records and review diffs to content hashes of native files, the system improves performance on tasks such as OfficeQA and APEX‑Agents, achieving higher pass rates and rubric scores compared to single‑view approaches. The study demonstrates that providing dual parsed/native access and visible diffs enhances agent performance, highlighting workspace state as a key experimental variable for future benchmarks.
By Yining Hua, Hongbin Na, Yifan Zhou, Akshay Kalose, Cyrus Ayubcha, Levi Lian
The paper "Runnable Commit Untangling for Coding Agents" introduces RucTangle, an agentic method that untangles large, tangled patches from coding agents while ensuring each commit keeps the code runnable. It also presents TangleEval, an evaluation framework that measures how untangled commit histories aid agents in repairing bugs. Experiments on 131 agent-generated patches show RucTangle produces fully runnable histories, outperforming baselines that leave 20.6%-37.4% of commits unrunnable, and improves bug‑repair success by 5.2% absolute in pass@1.
By Jinfeng Jiang, Dongsun Kim, Dayi Lin, Zhou Yang
arXiv:2608. 06701v1 Announce Type: cross Abstract: Fixing GitHub issues in large-scale projects is a long-horizon task, especially when a fix requires changes across multiple locations or the issue description lacks the information needed to localize and repair it.
By Shuyang Liu, Saman Dehghan, Ji Young Kim, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand
arXiv:2606. 12329v1 Announce Type: new Abstract: AI coding assistants now support a growing share of software work, from quick scripts to production applications.
By Ripon Chandra Malo, Tong Qiu
RefactorPlatform is an open‑source harness that standardizes the evaluation of repository‑scale refactoring agents by fixing the environment and systematically varying design choices such as model backbone, execution regime, and prompt specificity. Each run operates in an isolated workspace, logs detailed telemetry, and verifies changes with AST‑based checks. Experiments on 100 RefactorBench tasks show that AST‑aware chunking improves performance by 25‑30%, a lean retrieval‑augmented single agent outperforms a sub‑agent configuration, and retrieval’s accuracy gains offset its token overhead, keeping cost per successful refactoring unchanged.
By Aziz Ben Amor, Drish Mali, Mann Acharya, Vijayasri Iyer, S\'ebastien Brati\`eres
arXiv:2610.09633v1 Announce Type: new
Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engine...
By Igor Slinko, Yaroslav Golubev, Sergey Titov