The paper investigates hallucination in large language model–based automated program repair (APR). It defines hallucination as producing patches or intermediate artifacts that are not grounded in available repair evidence, and analyzes it across final patches and intermediate tasks such as triggering test case identification, line coverage prediction, and additional test case generation. Experiments on 832 Defects4J bugs show that only 21.0%–55.9% of patches pass the developer test suite, with 72.7% of sampled repairs exhibiting hallucinations, often due to incorrect causal localization or repair strategies.
By Xuemeng Cai, Jiakun Liu, Linhan Yang, Wei Ma, Lingxiao Jiang
SysML v2's textual syntax enables compiler-based validation of model structure and language conformance. However, semantic mistakes that preserve syntactic validity but violate domain rules cannot be detected through compilers.
arXiv:2606. 03628v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in open-ended text generation, yet they remain prone to hallucinating incorrect or unsupported content, which undermines their reliability.
By Lin Li, Georgia Channing, Suhaas M Bhat, Gabriel Davis Jones, Yarin Gal
arXiv:2606. 03331v1 Announce Type: cross Abstract: Consumer device repair is an important but underexplored testbed for large language models (LLMs).
By Atm Mizanur Rahman (University of Illinois Urbana-Champaign), Md Arid Hasan (University of Toronto), Syed Ishtiaque Ahmed (University of Toronto), Sharifa Sultana (University of Illinois Urbana-Champaign)
The paper investigates configuration problems in Retrieval-Augmented Generation (RAG) systems, which combine large language models with components like vector or graph databases. An empirical study of 654 real-world issues from 10 RAG systems categorizes symptoms, root causes, stages, and fix patterns, revealing that symptoms often appear early or late in the pipeline while root causes may lie elsewhere. The authors also offer nine actionable insights for practitioners to test, localize, and resolve configuration issues in RAG systems.
arXiv:2606. 05680v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled the automatic synthesis (generation) of register-transfer level (RTL) code from natural language instructions, offering a promising pathway to accelerate chip design.
By Mohammad Akyash, Nowfel Mashnoor, Kimia Azar, Hadi Kamali
The paper explores using a language model’s low‑level symbolic skill—specifically SQL—to detect hallucinations without fine‑tuning. By having the model construct an SQL database from reference documents, it can reason over both the source and the model’s output, creating a neurosymbolic check. Experiments on RAGTruth and DiaHalu show this method outperforms direct prediction and rivals state‑of‑the‑art detectors, highlighting the value of leveraging inherent symbolic competences in LLMs.
By Renato Vukovic, Hsien-chin Lin, Carel van Niekerk, Benjamin Ruppik, Michael Heck, Shutong Feng, Nurul Lubis, Milica Gasic
arXiv:2505. 07372v3 Announce Type: replace-cross Abstract: This paper presents a novel methodology for enhancing Automated Program Repair (APR) through synthetic data generation utilizing Large Language Models (LLMs).
By David de-Fitero-Dominguez, Antonio Garcia-Cabot, Eva Garcia-Lopez
arXiv:2610.15894v1 Announce Type: cross
Abstract: Large language model (LLM) code-repair systems fix faulty programs by repeatedly generating and testing revisions. Each revision is costly, making th...
By Elad Sharony, Ido Greenberg, Jialin Song, Shie Mannor, Gal Chechik, Eli Meirom
arXiv:2606. 20146v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly applied to computer-aided design (CAD) to generate design artifacts from textual instructions.
By Bharathi Kannan Nithyanantham, Clemens Kujat, Tobias Sesterhenn, Stefan Telgmann, J\"orn Pl\"onnigs, Stefan L\"udtke, Christian Bartelt
arXiv:2607. 06748v1 Announce Type: cross Abstract: Smart home automation platforms increasingly rely on user-authored YAML configuration files to define device behaviors, but these files are prone to syntax, formatting, and semantic logic errors that can cause automation failures and safety risks.
By Yizhi Wang, Xinghua Gao, Reachsak Ly, Alireza Shojaei
arXiv:2607. 10476v1 Announce Type: cross Abstract: Large language models (LLMs) have emerged as a powerful tool for retrieving knowledge through seamless, human-like interactions.
By Basel Abdeen, S M Tahmid Siddiqui, Meah Tahmeed Ahmed, Anoop Singhal, Latifur Khan, Punya Parag Modi, Ehab Al-Shaer