← Back to all news
arXiv AI

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

The paper investigates how large language models (LLMs) hallucinate nonexistent software packages during code generation and evaluates methods to mitigate this issue. It finds that current evaluation practices overestimate hallucination rates, especially for Python, and that Retrieval-Augmented Generation (RAG) and Self-Refine reduce hallucinations across multiple models and languages. The study also introduces Package Utility (PU) to measure whether defenses preserve useful recommendations and shows that Greedy decoding offers the best trade‑off between mitigation and utility, while adversarial prompts significantly increase hallucination rates, particularly in Ruby.

Related stories

arXiv AI
Aug 26

Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs

The paper introduces a two‑layer detector to prevent ‘slopsquatting’—the risk of local coding LLMs fabricating Python package names that adversaries can pre‑register on PyPI. The first layer checks PyPI for existence, while the second uses a Random Forest classifier on ten name‑and‑metadata features; an import reconciler resolves naming mismatches. Embedded in a LangGraph state machine, the system retries at escalating temperatures and falls back to stronger models, achieving hallucination‑free code in 76% of 300 curated prompts and recovering additional runs through intra‑ and cross‑model retries.

By Akash Raj, Sargam Sahu
llmscomputer-visionfine-tuningsafety
More like this →
arXiv Machine Learning
Sep 7

Harmless Yet Harmful: Neutral Prompting Attacks for Stealthy Hallucination Steering in Agent Skills

The paper introduces the Neutral Prompting Attack (NPA), a stealthy method that uses semantically benign instructions to increase the likelihood of large language models hallucinating non‑existent package names in coding agents. Unlike traditional dependency steering, NPA does not target a specific package but shifts the model’s output toward more speculative names. Experiments across multiple LLMs show that NPA raises hallucination rates, affects pip install success, alters the distribution of hallucinated packages, and bypasses existing static‑analysis, LLM‑based, and agent‑based defenses.

By Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Jun Sakuma
llmsagentsmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 15

SpecCloak: Probing Harmful Code Generation through Engineering Specifications

SpecCloak investigates how presenting coding tasks as engineering specifications—via the PseudoSpec template—affects large language models’ refusal to generate harmful code. Experiments on five open-weight models show that specification prompting significantly lowers refusal rates even when risk keywords remain, shifting internal representations toward code-output contexts. Building on this, SpecCloak combines PseudoSpec with discriminator-guided optimization to achieve a 28.91% attack success rate, while SpecReveal offers a defense that detects such attacks with 96.88% accuracy without fine‑tuning the target model.

By Wenbin Chen, Jiexin Wang, Guanting Wu, Jiaxi Liang, Yi Cai
llmsfine-tuningsafety
More like this →
Hugging Face Trending Papers
Jun 2

Learn from Your Mistakes: Tree-like Self-Play for Secure Code LLMs

While Large Language Models (LLMs) excel in code generation, they remain prone to replicating subtle yet critical vulnerabilities endemic to their training data. Current alignment techniques, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), typically apply coarse-grained optimization at the sequence level.

llmsreinforcement-learningfine-tuningbenchmarkssafety
More like this →