arXiv AI
Auditing Prompt Injection Defenses Before They Reach Users: What Strong Adversaries Reveal
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
6d ago
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
The paper examines the vulnerability of open‑weight large language model safeguards that aim to block harmful usage by assuming new harmful behavior arises only through fine‑tuning. It demonstrates that simple, low‑cost attacks—abliteration and prefilling—can significantly increase attack success rates across multiple benchmarks, and proposes an abliteration‑resistant tuning (ART) method that mitigates these attacks. The study shows that safeguarding defenses must be evaluated against a broader range of attack strategies beyond fine‑tuning.
arXiv AI
Jun 12
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
arXiv:2606. 12737v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources.
arXiv AI
Sep 15
ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents
arXiv:2609.14987v1 Announce Type: cross Abstract: Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prom...
arXiv Machine Learning
Jun 18
Lifecycle-Aware Dynamic Analysis for Secure ML Model Execution
arXiv:2606. 19023v1 Announce Type: cross Abstract: The growing reliance on pre-trained Machine Learning (ML) models has introduced new attack surfaces.
arXiv AI
Jun 4
TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering
arXiv:2602. 06911v2 Announce Type: replace-cross Abstract: As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, becomes critical to minimize risks.
arXiv Machine Learning
Jun 4
Bypassing Prompt Guards in Production with Controlled-Release Prompting
arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.