arXiv:2605. 17062v3 Announce Type: replace-cross Abstract: Spracklen et al.
By Aleksandr Churilov (Independent Researcher)
The paper investigates how large language models (LLMs) hallucinate nonexistent software packages during code generation and evaluates methods to mitigate this issue. It finds that current evaluation practices overestimate hallucination rates, especially for Python, and that Retrieval-Augmented Generation (RAG) and Self-Refine reduce hallucinations across multiple models and languages. The study also introduces Package Utility (PU) to measure whether defenses preserve useful recommendations and shows that Greedy decoding offers the best trade‑off between mitigation and utility, while adversarial prompts significantly increase hallucination rates, particularly in Ruby.
By Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, Tegawend\'e F. Bissyand\'e
The paper introduces the Neutral Prompting Attack (NPA), a stealthy method that uses semantically benign instructions to increase the likelihood of large language models hallucinating non‑existent package names in coding agents. Unlike traditional dependency steering, NPA does not target a specific package but shifts the model’s output toward more speculative names. Experiments across multiple LLMs show that NPA raises hallucination rates, affects pip install success, alters the distribution of hallucinated packages, and bypasses existing static‑analysis, LLM‑based, and agent‑based defenses.
By Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Jun Sakuma
arXiv:2509.22202v4 Announce Type: replace-cross
Abstract: Large language models (LLMs) now play a central role in code generation, yet they continue to hallucinate, frequently inventing non-existent...
By Lukas Twist, Mark Harman, Helen Yannakoudakis, Jie M. Zhang
arXiv:2608. 16187v1 Announce Type: cross Abstract: AI-assisted development tools generate vulnerable code at significant rates, yet few automated mechanisms exist to detect, enrich, fix, and verify security issues at development velocity, particularly ones that ground remediation in real-world threat context.
By Mikhail Surikov
The paper investigates hallucination in large language model–based automated program repair (APR). It defines hallucination as producing patches or intermediate artifacts that are not grounded in available repair evidence, and analyzes it across final patches and intermediate tasks such as triggering test case identification, line coverage prediction, and additional test case generation. Experiments on 832 Defects4J bugs show that only 21.0%–55.9% of patches pass the developer test suite, with 72.7% of sampled repairs exhibiting hallucinations, often due to incorrect causal localization or repair strategies.
By Xuemeng Cai, Jiakun Liu, Linhan Yang, Wei Ma, Lingxiao Jiang