arXiv AI

Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agents

arXiv:2607. 18826v1 Announce Type: cross Abstract: LLM-agent defenses are typically evaluated one session at a time.

arXiv AI

Jun 2

AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations

arXiv:2606. 02240v1 Announce Type: cross Abstract: Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls.

By Hiskias Dingeto, Will Leeney

llms agents multimodal benchmarks safety

arXiv AI

Jul 1

Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

arXiv:2606. 30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.

By Peizhi Niu, Wenjie Qu, Shangding Gu, Tianneng Shi, Yuankai Li, Ahmad Tawaha, Hend Alzahrani, Vincent Siu, Boyi Li, Chenguang Wang, Jiaheng Zhang, Basel Alomair, Ming Jin, Muhao Chen, Chi Wang, Costas Spanos, Dawn Song

llms agents benchmarks safety

arXiv Machine Learning

Jul 30

Adaptively Robust LLM Monitoring via Activation Watermarking

arXiv:2603. 23171v3 Announce Type: replace-cross Abstract: Providers monitor deployed large language models (LLMs) to detect misuse that they cannot prevent.

By Toluwani Aremu, Daniil Ognev, Samuele Poppi, Nils Lukas

llms fine-tuning benchmarks safety

arXiv AI

Jul 9

Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

arXiv:2607. 07474v1 Announce Type: cross Abstract: Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not.

By Harry Owiredu-Ashley

agents benchmarks safety

arXiv AI

Jun 16

Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems

arXiv:2606. 15242v1 Announce Type: cross Abstract: Skills are becoming the capability layer through which LLM agents turn plans into actions, but their use introduces security risks such as data leakage, unauthorized operations, and tool misuse.

By Yi Xie, Jiawei Du, Yu Cheng, Jiuan Zhou, Zhaoxia Yin

llms agents benchmarks

arXiv AI

Aug 11

ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners

arXiv:2608. 09732v1 Announce Type: cross Abstract: Agent skills are emerging as an important attack surface in LLM-based agent systems.

By Puyu Zeng, Simeng Qin, Jingzhi Li, Ju Jia, Zheli Liu, Xiaojun Jia

llms agents safety

arXiv AI

Jun 4

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

arXiv:2602. 06911v2 Announce Type: replace-cross Abstract: As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, becomes critical to minimize risks.

By Saad Hossain, Tom Tseng, Punya Syon Pandey, Samanvay Vajpayee, Matthew Kowal, Nayeema Nonta, Samuel Simko, Stephen Casper, Zhijing Jin, Kellin Pelrine, Sirisha Rambhatla

llms fine-tuning benchmarks safety

arXiv AI

Aug 10

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

arXiv:2608. 06909v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory.

By Jing Chen, Yang Sun, Li Zhang, Lin Xu, Jie Shi

llms agents benchmarks safety

arXiv AI

Jun 26

Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents

arXiv:2606. 26479v1 Announce Type: cross Abstract: Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a deterministic policy that mediates the agent's actions.

By Praneeth Narisetty, Shiva Nagendra Babu Kore, Uday Kumar Reddy Kattamanchi, Jayaram Kumarapu

llms agents benchmarks

arXiv AI

Jun 2

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

arXiv:2606. 00152v1 Announce Type: cross Abstract: LLM-based agents are rapidly advancing, autonomously invoking external tools to complete multi-step tasks for users.

By Mingxuan Zhang, Jiahui Han, Dadi Guo, Songze Li, Guanchu Wang, Na Zou, Dongrui Liu, Xia Hu

llms agents benchmarks safety

arXiv AI

Jul 21

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

arXiv:2607. 18063v1 Announce Type: cross Abstract: LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation.

By Devina Jain, David Hartmann, Chuan Li

llms agents robotics multimodal benchmarks safety

arXiv AI

Jun 6

From Attack Simulation to SIEM Rule: Deterministic Detection-as-Code Synthesis with Probe-Level Traceability

arXiv:2606. 05252v1 Announce Type: cross Abstract: Security teams routinely simulate attacks against their own systems to check whether their monitoring would catch a real intruder.

By Alexandre Cristov\~ao Maiorano

llms

Related stories