arXiv AI
Jun 12
Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests
arXiv:2606. 13449v1 Announce Type: cross Abstract: AI-agents (e.
Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset
arXiv:2606. 13468v1 Announce Type: cross Abstract: AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects.
Related stories
arXiv AI
Jul 21
AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
arXiv AI
Aug 10
Online Monitoring and Corrective Steering of Programming Agents
arXiv:2608. 06701v1 Announce Type: cross Abstract: Fixing GitHub issues in large-scale projects is a long-horizon task, especially when a fix requires changes across multiple locations or the issue description lacks the information needed to localize and repair it.
arXiv AI
Aug 5
Designing for Doubt: The Case for Informed Abstention in Autonomous Agents
arXiv:2606. 02965v2 Announce Type: replace Abstract: As large language models gain tool access and are deployed as autonomous agents capable of editing records, executing transactions, and modifying infrastructure, we still evaluate them based on the sole metric of task completion.
arXiv AI
Jun 3
What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents
arXiv:2606. 02965v1 Announce Type: new Abstract: Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.
arXiv AI
Jul 10
3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse
arXiv:2607. 07980v1 Announce Type: cross Abstract: Coding agents now author entire pull requests, and practitioners sharply disagree about what this does to code review: whether it becomes the bottleneck, whether human review is still necessary, and whether it quietly erodes the understanding that it once built.
arXiv AI
Aug 11
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.
arXiv Machine Learning
Jul 27
How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests
arXiv:2607. 21832v1 Announce Type: cross Abstract: Recent advances in large language models and their rapid adoption across software engineering tasks have made Artificial Intelligence (AI) coding agents an integral component of modern software development workflows.
arXiv AI
Jul 14
AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP
arXiv:2607. 11098v1 Announce Type: cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work.
arXiv AI
Aug 7
Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
arXiv:2608. 05179v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript drafting, and review.
arXiv Machine Learning
Jul 30
Can AI agents conduct open-ended AI research? Early evidence from two case studies
arXiv:2607. 27191v1 Announce Type: cross Abstract: Forecasts of explosive AI progress hinge on AI agents automating AI research.
arXiv AI
Jul 8
Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
arXiv:2607. 05775v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons.