arXiv Machine Learning

Harmless Yet Harmful: Neutral Prompting Attacks for Stealthy Hallucination Steering in Agent Skills

The paper introduces the Neutral Prompting Attack (NPA), a stealthy method that uses semantically benign instructions to increase the likelihood of large language models hallucinating non‑existent package names in coding agents. Unlike traditional dependency steering, NPA does not target a specific package but shifts the model’s output toward more speculative names. Experiments across multiple LLMs show that NPA raises hallucination rates, affects pip install success, alters the distribution of hallucinated packages, and bypasses existing static‑analysis, LLM‑based, and agent‑based defenses.

arXiv AI
Aug 25

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

The paper investigates how large language models (LLMs) hallucinate nonexistent software packages during code generation and evaluates methods to mitigate this issue. It finds that current evaluation practices overestimate hallucination rates, especially for Python, and that Retrieval-Augmented Generation (RAG) and Self-Refine reduce hallucinations across multiple models and languages. The study also introduces Package Utility (PU) to measure whether defenses preserve useful recommendations and shows that Greedy decoding offers the best trade‑off between mitigation and utility, while adversarial prompts significantly increase hallucination rates, particularly in Ruby.

By Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, Tegawend\'e F. Bissyand\'e
arXiv Machine Learning
3d ago

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

arXiv:2609.08040v1 Announce Type: cross Abstract: The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing d...

By Jiahao Shi, Edward Tsien, Yifeng Di, Hongjiao Zhang, Yuan Tang, Ronit Dey, Ilona Shishov, Gal Netanel, Zvi Grinberg, Vladimir Belousov, Bat-Zion Rotman, Ilan Pinto, Tianyi Zhang