arXiv AI
Zero2Repo: Can Coding Agents Build Repositories from Scratch?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
1d ago
E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
E2E-SWE is a benchmark that tests large language models’ ability to create complete, functional software repositories from scratch. It includes 186 tasks across 11 programming languages, each requiring an agent to build an installable project based solely on a natural‑language specification and an empty workspace, while passing a hidden test suite. The benchmark was crafted by software engineers and LLMs, then refined through iterative verification by autonomous agents to ensure clarity and solvability.
arXiv AI
Aug 11
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.
arXiv Machine Learning
Jul 30
REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
arXiv:2604. 01527v4 Announce Type: replace-cross Abstract: Production deployment of AI coding agents requires fast, reproducible evaluation signals.
arXiv Computation and Language
Sep 10
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
arXiv:2605.21384v2 Announce Type: replace-cross Abstract: As long-horizon coding agents produce more code than any developer can review, oversight collapses onto a single surface: the automated test...
arXiv Machine Learning
Jul 31
Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
arXiv:2607. 28591v1 Announce Type: cross Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation.
arXiv AI
Jun 3
Toward Training Superintelligent Software Agents through Self-Play SWE-RL
arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.