← Back to all news
arXiv AI By Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, Tegawend\'e F. Bissyand\'e

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

Read the original on arXiv AI →

The paper investigates how large language models (LLMs) hallucinate nonexistent software packages during code generation and evaluates methods to mitigate this issue. It finds that current evaluation practices overestimate hallucination rates, especially for Python, and that Retrieval-Augmented Generation (RAG) and Self-Refine reduce hallucinations across multiple models and languages. The study also introduces Package Utility (PU) to measure whether defenses preserve useful recommendations and shows that Greedy decoding offers the best trade‑off between mitigation and utility, while adversarial prompts significantly increase hallucination rates, particularly in Ruby.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 26

Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs

The paper introduces a two‑layer detector to prevent ‘slopsquatting’—the risk of local coding LLMs fabricating Python package names that adversaries can pre‑register on PyPI. The first layer checks PyPI for existence, while the second uses a Random Forest classifier on ten name‑and‑metadata features; an import reconciler resolves naming mismatches. Embedded in a LangGraph state machine, the system retries at escalating temperatures and falls back to stronger models, achieving hallucination‑free code in 76% of 300 curated prompts and recovering additional runs through intra‑ and cross‑model retries.

By Akash Raj, Sargam Sahu
llmscomputer-visionfine-tuningsafety
More like this →
arXiv Machine Learning
6d ago

Harmless Yet Harmful: Neutral Prompting Attacks for Stealthy Hallucination Steering in Agent Skills

The paper introduces the Neutral Prompting Attack (NPA), a stealthy method that uses semantically benign instructions to increase the likelihood of large language models hallucinating non‑existent package names in coding agents. Unlike traditional dependency steering, NPA does not target a specific package but shifts the model’s output toward more speculative names. Experiments across multiple LLMs show that NPA raises hallucination rates, affects pip install success, alters the distribution of hallucinated packages, and bypasses existing static‑analysis, LLM‑based, and agent‑based defenses.

By Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Jun Sakuma
llmsagentsmultimodalbenchmarkssafety
More like this →
Hugging Face Trending Papers
Jun 2

Learn from Your Mistakes: Tree-like Self-Play for Secure Code LLMs

While Large Language Models (LLMs) excel in code generation, they remain prone to replicating subtle yet critical vulnerabilities endemic to their training data. Current alignment techniques, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), typically apply coarse-grained optimization at the sequence level.

llmsreinforcement-learningfine-tuningbenchmarkssafety
More like this →