The paper presents a systematic analysis of five state‑of‑the‑art automated program repair agents, tracing their decision‑making across 500 real‑world repair tasks. It finds that while the agents perform well on simple fixes, they struggle with logic‑intensive bugs, often producing verbose, overfitted patches that pass tests without addressing root causes. Key bottlenecks identified include poor test generation, limited regression test selection, and reliance on primitive tooling without access to debuggers or advanced program analysis tools.
By Ira Ceka, Hailie Mitchell, Saurabh Pujar, Luca Buratti, Shyam Ramji, Junfeng Yang, Gail Kaiser, Baishakhi Ray
arXiv:2605. 17450v2 Announce Type: replace-cross Abstract: As software systems grow increasingly complex, automated vulnerability repair (AVR) remains difficult because the materials available to a repair system are usually failure artifacts rather than repair guidance.
By Simiao Liu, Fang Liu, Peiding Wang, Taichuan Li, Yinghao Zhu, Xiaoli Lian, Li Zhang
arXiv:2607. 12605v1 Announce Type: cross Abstract: Large language models (LLMs) have improved automated program repair (APR), but two limitations remain.
By Zhili Huang, Ling Xu, Hongyu Zhang
arXiv:2610.27784v1 Announce Type: cross
Abstract: Vulnerability benchmarks and software engineering research depend on rebuilding historical software. Yet builds rely on external resources, such as b...
By Olivier Nourry, Yoshiki Higo
arXiv:2607. 29422v1 Announce Type: cross Abstract: Automated vulnerability repair aims to reduce the time and effort required to patch security flaws from a vulnerability triage report.
By Michael Fu, Qiyue Mei, Patanamon Thongtanunam, Kla Tantithamthavorn
arXiv:2601.19066v3 Announce Type: replace-cross
Abstract: Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating fixes and aiding fix ge...
By Runxiang Cheng, Michele Tufano, Jos\'e Cambronero, Renyao Wei, Sherry Shi, Grant Uy, Pat Rondon, Franjo Ivan\v{c}i\'c
PatchBench introduces a benchmark to evaluate AI agents on realistic vulnerability patching tasks, addressing two key threats to validity: patch memorization and surface-level fixes that merely suppress crashes. The study finds that 25% of agent patches resemble historical developer patches, and that PoC-only validation inflates success rates by 1.83× on average. PatchBench mitigates these issues by selecting vulnerabilities whose true fixes lie outside the crash stack, migrating historical vulnerabilities into new contexts, and employing rigorous validation for security and semantic correctness.
By Chihao Shen, Jiacheng Li, Aastha Mahajan, Jeffery Siyuan Tian, Yonghwi Kwon, Yizheng Chen
arXiv:2608. 06410v1 Announce Type: new Abstract: Automated agent design improves agent harnesses through iterative revision, evaluation, and feedback summarization.
By Lekang Jiang, Bohan Tang, Stephan Goetz, Yiwen Guo
SWE‑Gate is a new repository‑level benchmark that evaluates software engineering agents on both functional correctness and review‑derived acceptance constraints. It creates 303 repair instances from real pull‑request review comments across 75 open‑source Python projects, providing separate functional and constraint tests along with compliant and non‑compliant patches. Experiments with four LLM backends show that while 644 repairs pass functional tests, 221 fail to meet the review constraints, highlighting a gap between functional success and full repair compliance.
By Xin He, Yanlin Wang, Mingwei Liu, Jiachi Chen, Hongyu Zhang, Guanbin Li
SkillScriptBench is a 350‑task benchmark that evaluates how well large language models can self‑evolve executable agent skill packages by distinguishing between documentation repair, script repair, and preservation. The benchmark draws from 100 real GitHub skill packages, creating 150 repair tasks with injected script faults and a controlled track of 200 tasks that test clean, documentation‑faulty, script‑faulty, and combined‑faulty states. Experiments show that AST‑Guided Skill Revision—using abstract syntax trees to target relevant code locations—improves repair success by up to 27.7% and yields more consistent outcomes across repeated runs.
whyItMatters":"The benchmark provides a systematic way to measure and improve the reliability of automated skill package maintenance, which is essential for deploying robust LLM agents in real‑world applications."
By Yuxuan Liu, Haoran Li, Yuhao Zhang, Jiahe Guo, Hongyu Luo, Wenbin Hu, Huihao Jing, Kawai Chung, Junle Chen, Changxuan Fan, Qing Zong, Lingyun Xie, Yangqiu Song
The paper introduces FlowAgent, an AI agent deployed at Google to automatically repair test failures in the pre-submit continuous integration workflow. FlowAgent uses a ReAct-style generate-and-validate loop with strict latency and quality filters, and was evaluated on 195 real-world failures with a 67.18% accuracy rate. After deployment, it suggested fixes on 295,508 changes, with developers previewing 65,069 and applying 28,554, and received positive feedback from interviews.
By Celal Ziftci, Spencer Greene, Ray Liu, Livio Dalloro, Lorenzo Dini
arXiv:2607. 18859v1 Announce Type: new Abstract: While Large Language Models have greatly advanced automated issue resolution, existing agent-based methods exhibit a fundamental limitation in their insufficient exploration of repair strategies.
By Tianyue Jiang, Yanlin Wang, Xin He, Daya Guo, Jiachi Chen, Ming Wen, Ensheng Shi, Xilin Liu, Yuchi Ma, Guanbin Li