arXiv AI
Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer
The paper explores whether natural‑language documentation aids coding agents in fixing software bugs and introduces a roundtrip benchmark that evaluates code descriptions by regenerating code and testing it. It finds that description completeness, not length, determines fidelity, and presents an optimizer that can produce fully faithful descriptions that generalize to new files. However, experiments across two model families and ten repositories show that such compact documentation does not improve an agent’s ability to resolve real repository issues compared to using the issue alone.
arXiv AI
Sep 1
Cost-Effective Repository Exploration for Agentic Issue Localization
arXiv:2608.29675v1 Announce Type: cross Abstract: Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository f...
Hugging Face Trending Papers
3d ago
LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems
Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks with large reference implementations. Many benchmarks...
arXiv Computation and Language
2d ago
LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems
arXiv:2609.37143v1 Announce Type: cross Abstract: Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks wi...
arXiv AI
2d ago
Is Agent Code Less Maintainable Than Human Code?
arXiv:2606.21804v2 Announce Type: replace-cross Abstract: Maintainability is a core dimension of software engineering, shaping how code is written, reviewed, and developed over time. While coding age...
arXiv AI
1d ago
E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
E2E-SWE is a benchmark that tests large language models’ ability to create complete, functional software repositories from scratch. It includes 186 tasks across 11 programming languages, each requiring an agent to build an installable project based solely on a natural‑language specification and an empty workspace, while passing a hidden test suite. The benchmark was crafted by software engineers and LLMs, then refined through iterative verification by autonomous agents to ensure clarity and solvability.