The paper introduces a pre‑action verification framework for large language model agents that emit shell commands or code edits. By running deterministic checks before execution, the system can catch invalid commands (95.8% success with a 10.0% false‑positive rate) and prevent silent failures in code edits, achieving high recall while minimizing false positives. The authors provide benchmarks, verifiers, and guards for both action modalities.
By Asaad Althoubi
arXiv:2609.13728v1 Announce Type: cross
Abstract: Historical Linux kernel patches capture defect knowledge that applies beyond their original repair sites. Recent work has shown that large language m...
By Ruoyu Wang, Tuo Li, Jia Li
arXiv:2608. 14863v1 Announce Type: cross Abstract: LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified.
By Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang, Shaoyu Wang, Huanchen Sun, Seo Jin Park
The paper examines how large‑language‑model (LLM) decompilers, which produce clean, idiomatic C code, are currently evaluated mainly on recompilability and passing shipped tests. It shows that these metrics can mask significant behavioral differences: a decompiled function may recompile and pass all tests yet diverge on other inputs or lose disclosed vulnerabilities. To address this, the authors propose Decompile‑Diverge, a behavioral oracle that synthesizes drivers, fuzzes inputs, and compares the decompiled code’s behavior to the original, revealing divergences in up to 13% of cases and exposing gaps in current evaluation suites.
By Chang Liu, Edward Raff, Kristopher Micinski
The paper investigates how large language models (LLMs) handle bug fixing versus problem solving in competitive programming. Using a dataset of ~3,000 Codeforces submissions and their human fixes, the authors compare LLM-generated patches to human patches and assess whether LLMs prefer to modify buggy code or generate new solutions. Results show that LLMs often alter more lines than necessary and sometimes produce entirely new solutions, performing better when allowed to generate solutions from scratch rather than patching existing code.
By Alexandru Stefan Stoica, Traian Rebedea, Marian Cristian Mihaescu
The paper investigates how Large Language Models (LLMs) handle bug fixing compared to human-written patches by analyzing about 3,000 Codeforces submissions. It finds that LLMs often modify more lines than necessary and sometimes produce entirely new solutions, and that they solve more problems correctly when generating solutions from scratch rather than patching existing code. The study highlights implications for AI‑assisted programming tools, suggesting a shift toward incremental problem‑solving strategies.
arXiv:2607. 06411v1 Announce Type: cross Abstract: Developers increasingly delegate real maintenance work to product-grade coding agents, and many state tasks in their native language, in the style of a customer request rather than a curated English issue.
By Evgeny Shilov (Independent Researcher)
arXiv:2609.10123v1 Announce Type: cross
Abstract: Large language models (LLMs) have become ubiquitous in software development, with LLM-based automated program repair tools increasingly used during c...
By Xietao Wang-Lin, Anton Isopoussu, Louis Mahon
arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.
By Xin Zhou, Chun Yong Chong, Kisub Kim, Yun Peng, Rui Shu, Zihan Wu, Xu Han, Guowen Yuan, Zeyang Zhuang, Jounghoon Kim, Jeongjin Ju, Seongmin Ju, Taein Yoon, David Lo
arXiv:2607. 11098v1 Announce Type: cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work.
By Aritra Mazumder, Nusrat jahan Lia
arXiv:2610.12569v1 Announce Type: new
Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue th...
By Wei Minn, Zhensu Sun, Lwin Khin Shar, David Lo
arXiv:2610.13563v1 Announce Type: cross
Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,...
By Han Peng, Xing Hu, Feifei Niu, Huan Li, Xin Xia