arXiv AI

StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection

arXiv:2608. 06477v1 Announce Type: cross Abstract: Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages.

arXiv AI

Jun 2

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

arXiv:2603. 24511v2 Announce Type: replace-cross Abstract: We show that AI agents are capable of discovering novel algorithms for adversarial attacks against LLMs, advancing the state of the art on white-box jailbreaking and prompt injection evaluations.

By Alexander Panfilov, Peter Romov, Igor Shilov, Yves-Alexandre de Montjoye, Jonas Geiping, Maksym Andriushchenko

llms agents multimodal benchmarks safety

arXiv AI

Jul 21

STAC: When Innocent Tools Form Dangerous Chains for LLM Agents

arXiv:2509. 25624v3 Announce Type: replace-cross Abstract: As LLMs advance into autonomous agents with tool-use capabilities, they introduce security challenges that extend beyond traditional content-based LLM safety concerns.

By Jing-Jing Li, Jianfeng He, Chao Shang, Devang Kulshreshtha, Xun Xian, Yi Zhang, Hang Su, Sandesh Swamy, Yanjun Qi

llms agents multimodal benchmarks safety

arXiv AI

Jun 16

AutoDojo: Adaptive Attacks Expose Superficial Defenses and User-Underspecification Limits in LLM Agents

arXiv:2606. 15057v1 Announce Type: cross Abstract: Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.

By Xinhang Ma, Taoran Li, Chaowei Xiao, Zhiyuan Yu, Ning Zhang, Yevgeniy Vorobeychik

llms agents multimodal benchmarks

arXiv AI

Jul 1

Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

arXiv:2606. 30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.

By Peizhi Niu, Wenjie Qu, Shangding Gu, Tianneng Shi, Yuankai Li, Ahmad Tawaha, Hend Alzahrani, Vincent Siu, Boyi Li, Chenguang Wang, Jiaheng Zhang, Basel Alomair, Ming Jin, Muhao Chen, Chi Wang, Costas Spanos, Dawn Song

llms agents benchmarks safety

arXiv AI

Jun 15

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

arXiv:2606. 13994v1 Announce Type: cross Abstract: LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world.

By Vikhyath Kothamasu, Virginia Smith, Chhavi Yadav

llms agents benchmarks safety

arXiv AI

Jun 15

From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails

arXiv:2606. 14517v1 Announce Type: cross Abstract: LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents.

By Yuguang Zhou, Xunguang Wang, Pingchuan Ma, Zhantong Xue, Zhaoyu Wang, Shuai Wang

llms agents benchmarks safety

arXiv AI

Jun 6

CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

arXiv:2601. 09923v3 Announce Type: replace Abstract: AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior.

By Hanna Foerster, Tom Blanchard, Kristina Nikoli\'c, Ilia Shumailov, Cheng Zhang, Robert Mullins, Nicolas Papernot, Florian Tram\`er, Yiren Zhao

agents

arXiv AI

Jun 6

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

arXiv:2606. 05233v1 Announce Type: cross Abstract: Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel.

By Nicholas Saban

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Aug 11

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents

arXiv:2608. 09476v1 Announce Type: cross Abstract: Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API.

By Hongwei Yao, Yiming Liu, Meihui Chen, Jieling Chen, Zikun Chen, Yiling He, Wangze Ni, Cong Wang, Kui Ren

llms agents benchmarks safety

arXiv Machine Learning

Jul 30

AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents

arXiv:2607. 26998v1 Announce Type: cross Abstract: Large language model (LLM) agents automate penetration testing through an observation-action loop, selecting actions based on observations returned by tools.

By Ruoyu Wang, Heng Zhao, Renjie Wu, Mengnan Zhao, Zhixuan Chu, Wanyu Lin, Tianhang Zheng

llms agents

arXiv AI

Jun 16

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.

By Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan

llms rag agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 30

Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense

arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.

By Subhadip Mitra

llms benchmarks safety

Related stories