arXiv AI

Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer

The paper explores whether natural‑language documentation aids coding agents in fixing software bugs and introduces a roundtrip benchmark that evaluates code descriptions by regenerating code and testing it. It finds that description completeness, not length, determines fidelity, and presents an optimizer that can produce fully faithful descriptions that generalize to new files. However, experiments across two model families and ten repositories show that such compact documentation does not improve an agent’s ability to resolve real repository issues compared to using the issue alone.

Hugging Face Trending Papers
Aug 20

From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation

The study investigates how autonomous coding agents interact with technical documentation, analyzing 557 coding sessions and 33,097 pull requests. Findings reveal that agents primarily engage with agent-facing artefacts, show weak links between documentation consultation and code editing, lack explicit validation sequences, and tend to consult documentation after code changes. The authors propose a two‑lobed cycle model of agent‑documentation interaction and challenge assumptions about actionability and verifiability of agent‑friendly documentation.

arXiv AI
Jul 22

Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality

arXiv:2607. 03691v2 Announce Type: replace-cross Abstract: Coding agents, autonomous systems that use large language models (LLMs) to resolve software engineering tasks, rely on agent harness: a middleware layer in between a developer and a large language model that orchestrates system prompts, tool execution, context management, and iterative reasoning loops.

By Oussama Ben Sghaier, Hao Li, Bram Adams, Ahmed E. Hassan
arXiv AI
Aug 11

A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents

arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.

By Xin Zhou, Chun Yong Chong, Kisub Kim, Yun Peng, Rui Shu, Zihan Wu, Xu Han, Guowen Yuan, Zeyang Zhuang, Jounghoon Kim, Jeongjin Ju, Seongmin Ju, Taein Yoon, David Lo