arXiv AI

Externalizing Requirement-to-Repair Artifacts as Observable Traces for LLM-Based Program Repair

The paper introduces THEMIS, a stage-aware repair workflow that externalizes the requirement-to-repair process by generating semantic interpretations, a runtime requirement-code graph, graph-derived developer guidance, retained repair rationale and patches, and post-edit audit records. A retrospective audit of 300 SWE-bench Lite cases shows that these artifacts enable cross-stage inspection, with a complete developer rationale available for 288 cases and 214 cases retaining a full audited field set. The retained records also allow systematic measurement of cross-stage correspondence, revealing high recurrence of target symbols across rationales and patches, and a preliminary improvement in resolving cases compared to a direct same-input condition.

arXiv AI
Sep 7

Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program Repair

The paper investigates hallucination in large language model–based automated program repair (APR). It defines hallucination as producing patches or intermediate artifacts that are not grounded in available repair evidence, and analyzes it across final patches and intermediate tasks such as triggering test case identification, line coverage prediction, and additional test case generation. Experiments on 832 Defects4J bugs show that only 21.0%–55.9% of patches pass the developer test suite, with 72.7% of sampled repairs exhibiting hallucinations, often due to incorrect causal localization or repair strategies.

By Xuemeng Cai, Jiakun Liu, Linhan Yang, Wei Ma, Lingxiao Jiang
arXiv AI
Jul 31

TraceCoder: Explainable and Auditable Code Generation with Position-Key Snippet Versioning

arXiv:2607. 26307v1 Announce Type: new Abstract: Contemporary LLM-based coding agents produce code as black-box outputs: the rationale behind each line is hidden, the evolution of the code through benchmark-driven repair is ephemeral, and post-hoc auditing is impossible.

By Rwaida Alssadi, Muntaser Syed, Balaji Kasula, Lamine Deen, Majed Alotaibi, Mohammed Alghamdi, Tyler Ton, Ali Alqarni, Marius Silaghi
arXiv AI
Sep 7

The History Is the Detector: Executing CVE Patch History, End-to-End

The paper introduces BUGSTONE‑E2E, a framework that converts vulnerability history into executable detection rules and validates them. It mines reusable rules from fixing commits, organizes them by CWE and language, and applies a funnel‑shaped pipeline that starts with lightweight analysis and culminates in LLM‑guided inspection, runtime verification, and patch generation. Using 19,325 high‑severity CVEs, the system identified 2,710 fixing commits, created 1,033 detection rules across 56 CWE families, and produced runtime evidence for 644 findings in 14 programs.

By Qiushi Wu, Kevin Eykholt, Youngja Park, Xiaokui Shu, Dhilung Kirat, Douglas Lee Schales, Ian Molloy
arXiv AI
Sep 4

SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents

SWE‑Gate is a new repository‑level benchmark that evaluates software engineering agents on both functional correctness and review‑derived acceptance constraints. It creates 303 repair instances from real pull‑request review comments across 75 open‑source Python projects, providing separate functional and constraint tests along with compliant and non‑compliant patches. Experiments with four LLM backends show that while 644 repairs pass functional tests, 221 fail to meet the review constraints, highlighting a gap between functional success and full repair compliance.

By Xin He, Yanlin Wang, Mingwei Liu, Jiachi Chen, Hongyu Zhang, Guanbin Li