arXiv AI July 31, 2026 By Lehan Wang, Boli Chen, Ruixue Ding, Pengjun Xie, Jinwei Huang, Zhendong Liu, Shuo Wang, Tao Lei, Xin Ouyang, Xiaomeng Li

SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response

Read the original on arXiv AI →

arXiv:2607. 26791v1 Announce Type: cross Abstract: Large Language Model (LLM) agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces (CLIs), making it critical to thoroughly assess their security capabilities.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 15

AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Ranges

arXiv:2606. 14295v1 Announce Type: cross Abstract: Frontier AI systems are increasingly capable of cybersecurity tasks, including codebase inspection, vulnerability detection, and exploitation.

By Fengyu Liu, Jiarun Dai, Yihe Fan, Wuyuao Mai, Ziao Li, Bofei Chen, Jie Zhang, Zheng Lou, Bocheng Xiang, Qiyi Zhang, Xudong Pan, Geng Hong, Yuan Zhang, Min Yang

llms agents benchmarks

arXiv AI

Jun 12

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.

By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang

llms agents safety

arXiv AI

Jul 1

Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

arXiv:2606. 30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.

By Peizhi Niu, Wenjie Qu, Shangding Gu, Tianneng Shi, Yuankai Li, Ahmad Tawaha, Hend Alzahrani, Vincent Siu, Boyi Li, Chenguang Wang, Jiaheng Zhang, Basel Alomair, Ming Jin, Muhao Chen, Chi Wang, Costas Spanos, Dawn Song

llms agents benchmarks safety

arXiv AI

Jul 29

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

arXiv:2607. 25379v1 Announce Type: new Abstract: Cyber-capable AI agents combine language models with tools, memory, and execution en- vironments to perform multi-step offensive-security tasks.

By Abu Bakar Siddik

llms agents

arXiv Machine Learning

3d ago

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

arXiv:2508.14925v2 Announce Type: replace-cross Abstract: By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a...

By Zhiqiang Wang, Yichao Gao, Yanting Wang, Suyuan Liu, Haifeng Sun, Haoran Cheng, Guanquan Shi, Haohua Du, Xiangyang Li

llms agents benchmarks safety

arXiv AI

Jun 2

AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations

arXiv:2606. 02240v1 Announce Type: cross Abstract: Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls.

By Hiskias Dingeto, Will Leeney

llms agents multimodal benchmarks safety

One email a morning, machine-written

Related stories

AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Ranges

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations