← Back to all news
arXiv Machine Learning By Amelia Kawasaki, Stefan Lee

Attention-based Classification for Indirect Prompt Injection

Read the original on arXiv Machine Learning →

The paper evaluates the Attention Tracker, a white‑box detection method for indirect prompt injection (IPI) attacks, in realistic agentic settings using the AgentDojo benchmark. It finds that the method’s claimed dataset‑agnostic detection fails in agentic environments, with near‑chance performance across most model families. The authors propose AttnDA, a Fisher Linear Discriminant Analysis classifier that replaces the scalar threshold, which improves detection across multiple benchmarks but still struggles to transfer to AgentDojo, highlighting the importance of training data from the target domain.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 26

What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions

The paper introduces Attnlocate, a runtime framework that localizes behavior‑guiding instructions within the attention matrix of large language model agents. By treating this localization as an object detection task, Attnlocate uses a multi‑head, multi‑layer attention aggregation scheme and a 1‑D U‑Net to identify spans that influence tool‑calling decisions. The system then adjudicates potential malicious invocations based on the authority of the source, achieving high detection metrics across diverse LLM families and demonstrating transferability to unseen models.

By Yichao Gao, Yumo Zhang, Yunhao Yao, Haohua Du, Puhan Luo, Ruiqi Li, Zhiqiang Wang
llmsagentscomputer-vision
More like this →