arXiv Computation and Language

AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans

arXiv AI
Oct 26

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

The study surveys popular, active GitHub repositories across thirteen languages, treating each repository’s AGENTS.md and CLAUDE.md files as instruction sets for coding agents. It finds that most repositories include such instructions, with CLAUDE.md often referencing AGENTS.md, and that these files contain a mix of project knowledge and workflow rules, including varying stances on AI attribution. The analysis reveals that more popular projects tend to have stricter pull‑request rules and AI contribution guidelines, and that LLM and agent projects are over‑represented among adopters.

By Pawel Zarzycki
arXiv AI
Oct 31

How Do Developers Edit AI-Generated Code, and How Well Do LLMs Infer Their Edits? Insights from 53.6K Developer Edits

The paper introduces DECODE, a dataset of 53.6K real‑world in‑IDE edits of AI‑generated code from over 1,000 developers, and uses it to study when, why, and how developers modify such code. Findings show developers spend little effort editing and often abandon AI outputs, highlighting the need for LLMs to infer intent from edits. Experiments reveal that while frontier models can generate edits that align with developer intent, they tend to over‑edit and revert changes; fine‑tuning improves performance but does not fully close the gap.

By Jenny T. Liang, Mihika Bairathi, Wayne Chi, Ameet Talwalkar, Nishant Subramani, Valerie Chen
Hugging Face Trending Papers
Jun 23

Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories

Generative AI coding agents are entering the open-source supply chain, yet their diverse and often invisible traces leave their prevalence poorly understood. We introduce a multi-layered detection framework that integrates configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup across World of Code (180M+ Git repositories), classifying agent traces into four behavioral types.

Hugging Face Trending Papers
Aug 20

From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation

The study investigates how autonomous coding agents interact with technical documentation, analyzing 557 coding sessions and 33,097 pull requests. Findings reveal that agents primarily engage with agent-facing artefacts, show weak links between documentation consultation and code editing, lack explicit validation sequences, and tend to consult documentation after code changes. The authors propose a two‑lobed cycle model of agent‑documentation interaction and challenge assumptions about actionability and verifiability of agent‑friendly documentation.

Hugging Face Trending Papers
Jul 15

Early Adoption of Agentic Coding Tools by GitHub Projects

Agentic coding tools are increasingly capable of generating and submitting pull requests (PRs) to software projects, introducing new forms of human-agent collaboration in software development. While prior studies have examined PR-level outcomes of agent-generated contributions, less is known about how agentic coding tools are adopted and managed at the project level.

arXiv AI
Jul 22

Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality

arXiv:2607. 03691v2 Announce Type: replace-cross Abstract: Coding agents, autonomous systems that use large language models (LLMs) to resolve software engineering tasks, rely on agent harness: a middleware layer in between a developer and a large language model that orchestrates system prompts, tool execution, context management, and iterative reasoning loops.

By Oussama Ben Sghaier, Hao Li, Bram Adams, Ahmed E. Hassan