The study surveys popular, active GitHub repositories across thirteen languages, treating each repository’s AGENTS.md and CLAUDE.md files as instruction sets for coding agents. It finds that most repositories include such instructions, with CLAUDE.md often referencing AGENTS.md, and that these files contain a mix of project knowledge and workflow rules, including varying stances on AI attribution. The analysis reveals that more popular projects tend to have stricter pull‑request rules and AI contribution guidelines, and that LLM and agent projects are over‑represented among adopters.
By Pawel Zarzycki
SlackBench is a new benchmark that evaluates AI agents on realistic collaborative research and engineering workflows, requiring joint reasoning over workplace communication and evolving repository state. The benchmark is built from privacy‑preserving reenactments of real academic projects, providing reproducible environments with multi‑party dialogue, code changes from GitHub, and Google Docs. It contains 140 queries covering heterogeneous‑source search, abstention, summarization, and inference over long conversations, and shows that current models perform well below the ceiling, indicating significant room for improvement.
By Mingxuan Li, Huanzhi Mao, Ruitao Zheng, Jarod Stanbury James, Yunze Li, Natanya Anderson, Joseph E. Gonzalez, Alex Dimakis
arXiv:2610.07832v1 Announce Type: cross
Abstract: Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, e...
By Haibo Jin, Xinjie Li, Peng Kuang, Haohan Wang
arXiv:2507. 16395v3 Announce Type: replace Abstract: Atomic commits, which address a single development concern, are a best practice in software development.
By Bo Hou, Xin Tan, Kai Zheng, Fang Liu, Yinghao Zhu, Li Zhang
The paper introduces ReqClarifyBench, an interactive benchmark for evaluating how well large language models (LLMs) can clarify vague user intent into clear requirements in repository-level software development. It includes 341 tasks across game, miniapp, and web development, each with a vague intent, a human-validated requirement document, and code evaluation artifacts, and uses LLM-based user simulators with diverse personas. The study evaluates six LLMs on clarification effectiveness and requirement quality, finding that stronger clarification does not always yield better documents, performance varies by persona, and coding-agent harnesses do not improve document quality over base LLMs.
By Mingyu Chen, Zhiqing Zhong, Manyi Wang, Linxi Liang, Rouyu Chen, Pinjia He
Skill Dreaming is a framework that learns from pull request (PR) streams to build and continuously refine a repository‑specific skill library. By extracting executable knowledge from validated PRs, it guides LLM agents in implementing features, maintaining tests, documentation, and repository conventions. The framework integrates PR analysis, skill management, code execution, and continuous integration to create an end‑to‑end workflow that updates skills as new PRs arrive.
By Xinyuan Song, Hasibul Haque, Jay Somasundaram, Ryan Turner, Bowen Zhu, Liang Zhao
arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.
By Xin Zhou, Chun Yong Chong, Kisub Kim, Yun Peng, Rui Shu, Zihan Wu, Xu Han, Guowen Yuan, Zeyang Zhuang, Jounghoon Kim, Jeongjin Ju, Seongmin Ju, Taein Yoon, David Lo
arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.
By Mohit Raghavendra, Anisha Gunjal, Aakash Sabharwal, Yunzhong He
SkillForge is a self‑distillation framework that proactively builds project‑specific knowledge for large language model agents by synthesizing and resolving artificial issues derived from a repository’s test‑covered core functionalities. By distilling these solutions into entity‑grounded skills linked to repository entities, the system equips agents with reusable, project‑specific expertise before encountering real issues. Experiments with both open‑source and closed‑source models show that SkillForge consistently outperforms strong baselines in issue resolution tasks.
By Silin Chen, Han Li, Xiaodong Gu, Yuling Shi, Haibing Guan
arXiv:2607. 27146v1 Announce Type: cross Abstract: Coding agents have made substantial progress on software engineering tasks that modify existing codebases, including bug fixing and feature implementation.
By Yihao Chen, Shi Chang, Khaled Chawa, Feng Lin, Boyuan Chen, Shaowei Wang, Ahmed E. Hassan
SkillForge is a self‑distillation framework that proactively builds project‑specific knowledge for large language model agents by synthesizing and resolving artificial issues derived from a repository’s test‑covered core functionalities. Rather than waiting for real issues to reveal knowledge gaps, SkillForge generates these synthetic problems, learns reusable entity‑grounded skills, and associates them with relevant repository entities. Experiments with both open‑source and closed‑source models show that this proactive knowledge acquisition consistently outperforms strong baselines in issue resolution tasks.
arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.
By Yuxiang Wei, Zhiqing Sun, Emily McMilin, Jonas Gehring, David Zhang, Gabriel Synnaeve, Daniel Fried, Lingming Zhang, Sida Wang