The study examines whether the CodeHealth metric can predict the token cost of agentic code review and fix for pull requests. Using a pre‑registered observational design, 465 agent‑authored PRs were processed by a five‑agent pipeline across three LLMs, and mixed models showed that each CodeHealth point corresponded to a 3.5%–6.1% reduction in token usage, independent of PR footprint. The effect was strongest during the review stage and appears mediated by file size rather than PR size.
By Lo Gullstrand Heander, Markus Borg, Emma S\"oderberg, Christofer Rydenf\"alt
arXiv:2606.28235v2 Announce Type: replace-cross
Abstract: In July 2026, about 1,200 AI agents in an internal OpenAI evaluation, meant to run in isolation, used a shared package server as a message bo...
By Daniel Russo
arXiv:2606. 18168v1 Announce Type: cross Abstract: Software practitioners increasingly use AI coding agents that generate test code alongside production code in open source pull requests (PRs).
By Dipayan Banik, Kowshik Chowdhury, Shazibul Islam Shamim
arXiv:2608. 20195v1 Announce Type: cross Abstract: Technical documentation is written for human developers, but an increasing share of software changes is now authored by autonomous coding agents.
By Zhijun Gao, Jing Chen
VeriHarness is a method that enhances verification for large language model agents tackling long‑horizon tasks without needing reference answers at test time. It transforms the base LLM into an agentic verifier by providing a workspace, evidence tools, and reusable verification skills, using disagreement resolution and consensus challenge to evaluate competing claims. Across five benchmarks and two frontier models, VeriHarness outperforms baselines, achieving significant performance gains and demonstrating self‑improvement of verification skills from failure feedback.
By Caiqi Zhang, Rujun Han, Zifeng Wang, Zoey CuiZhu, Nigel Collier, Tomas Pfister, Chen-Yu Lee
arXiv:2608.29464v1 Announce Type: cross
Abstract: Chain-of-thought (CoT) monitoring assumes that reasoning traces faithfully record the information that shapes a model's answer. Existing faithfulness...
By Aryo Pradipta Gema, Neel Rajani, Rohit Saxena, Wai-Chung Kwan, Pasquale Minervini
The paper investigates how language‑model judges can make version‑dependent errors when evaluating upgraded agents. Using 35 public coding‑agent submissions, two customer‑service agents, and over a thousand expert‑labeled trajectories, the authors show that fixed judges often reject task‑conditioned error invariance and can incorrectly approve failed patches, especially as agent capability increases. Paired audits of current outputs reduce interval width only marginally, and the study concludes that independent human patch review is still necessary.
By Jiapeng Li
arXiv:2606. 27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the strength and scope of the supporting evidence.
By Yu Fu, Yongqi Kang, Yong Zhao
SWE-Prometheus is a new benchmark that evaluates large language model coding agents on the broader task of improving repository engineering governance, rather than just fixing individual issues. It presents fixed snapshots with open-ended objectives, requiring agents to identify risks, prioritize interventions, and verify changes across six governance dimensions. The benchmark includes 60 repositories and reports metrics such as Normalized Governance Improvement and behavior‑breakage rates, providing a nuanced view of how different models affect governance artifacts and execution‑backed improvements.
By Jiajun Wu, Leixin Sun, Zihan Tan, Yitao Liu, Shuo Li, Jiaru Qian, Shanghaoran Quan, Chuangxin Zhao, Yangxu Liao, Yang Liu, Bin Chong, Guancheng Wan
The study investigates how autonomous coding agents interact with technical documentation, analyzing 557 coding sessions and 33,097 pull requests. Findings reveal that agents primarily engage with agent-facing artefacts, show weak links between documentation consultation and code editing, lack explicit validation sequences, and tend to consult documentation after code changes. The authors propose a two‑lobed cycle model of agent‑documentation interaction and challenge assumptions about actionability and verifiability of agent‑friendly documentation.
The study investigates whether adding specialist language‑model contexts can enhance software assurance when deterministic tool evidence is already available. Five experiments were conducted using Qwen2.5‑Coder‑7B‑Instruct across Java warning controls, OWASP Benchmark, Defects4J test generation, SV‑COMP assertion verdicts, and misleading evidence packages. Results show that increasing context count does not improve unsafe clearance, that verifier feedback can raise correct verdicts but also incorrect ones, and that models reject most counterexamples while scope checks classify packages accurately, supporting deterministic evidence admission but highlighting limits in verdict aggregation and feedback interpretation.
By S M Abdulla Al Mamun, Akif Islam, Mohd. Ruhul Ameen
arXiv:2606. 13468v1 Announce Type: cross Abstract: AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects.
By Mahmoud Abujadallah, Ali Arabat, Mohammed Sayagh