arXiv Computation and Language September 28, 2026 By Jiawen Wang, Pritha Gupta, Eyke H\"ullermeier, Xiaoxue Gao, Nancy F. Chen

From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMs

Read the original on arXiv Computation and Language →

The paper investigates prompt injection attacks on 14 open‑source and 3 closed‑source large language models (LLMs), introducing a new metric called Attack Success Probability (ASP) that accounts for uncertainty in model responses. It demonstrates that a simple hypnotism attack can trigger objectionable behavior in models such as StableLM2, Mistral, Openchat, and Vicuna, achieving roughly 90% ASP. The study highlights that moderately well‑known LLMs are particularly vulnerable, underscoring the importance of public awareness and effective mitigation strategies.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 4

Bypassing Prompt Guards in Production with Controlled-Release Prompting

arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.

By Jaiden Fairoze, Sanjam Garg, Keewoo Lee, Mingyuan Wang

llms safety

arXiv Machine Learning

Aug 11

BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes

arXiv:2608. 08027v1 Announce Type: cross Abstract: Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data.

By Laiqiao Qin, Tianqing Zhu, Longxiang Gao, Wanlei Zhou

llms rag

arXiv Machine Learning

Jun 30

FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks

arXiv:2606. 28962v1 Announce Type: cross Abstract: Model quantization is essential for the efficient deployment of Large Language Models (LLMs), but introduces a critical vulnerability: Quantization-Conditioned Backdoor (QCB) attacks.

By Aoying Zheng, Anqi Du, Zizhuang Deng, Yuxuan Chen

llms efficiency safety

arXiv AI

Jun 16

Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

arXiv:2605. 26595v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison.

By Zedian Shao, Charles Fleming, Teodora Baluta

llms fine-tuning

arXiv AI

Jun 19

Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems

arXiv:2606. 20470v1 Announce Type: cross Abstract: Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents.

By Reza Soosahabi, Vivek Namsani

agents multimodal benchmarks safety

arXiv Computation and Language

Aug 27

A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.

By Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman, Ali Hassan, Umar Mujahid

llms rag fine-tuning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Bypassing Prompt Guards in Production with Controlled-Release Prompting

BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes

FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks

Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems

A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots