Securing Multi-Agent Systems by Design
Multi-agent systems solve complex tasks by decomposing them into subtasks, delegating the subtasks to agents, and aggregating the agents' responses using an aggregation rule. Malicious agents can mani...
The paper introduces response manipulation attacks (ReMA) that iteratively optimize malicious agent responses to compromise multi‑agent system outputs. It proposes SecMAS, a secure aggregation rule that selects responses most consistent with a large subset of others, providing theoretical security guarantees against both untargeted and targeted attacks. Experiments show existing aggregation rules are vulnerable to ReMA, while SecMAS defends effectively without sacrificing utility.
Multi-agent systems solve complex tasks by decomposing them into subtasks, delegating the subtasks to agents, and aggregating the agents' responses using an aggregation rule. Malicious agents can mani...
arXiv:2508. 16481v3 Announce Type: replace Abstract: Ensuring the safe use of agentic systems requires a thorough understanding of the range of malicious behaviors these systems may exhibit.
arXiv:2606. 24496v1 Announce Type: cross Abstract: The use of agentic systems to perform offensive security operations has moved from a theoretical possibility to a commoditized capability.
arXiv:2606. 02302v1 Announce Type: cross Abstract: Autonomous LLM agents increasingly operate in stateful environments where they access tools, files, memory, and external services.
The use of agentic systems to perform offensive security operations has moved from a theoretical possibility to a commoditized capability. However, while the community has focused on creating more and more capable agents, less attention has been allocated to assessing the security of those systems.
MiniRep is a reputation‑based aggregation system designed for multi‑agent debate (MAD) that remains robust even when malicious agents are present. It evaluates agents on both their current task performance and historical reputation, while preventing groups of agents with highly similar responses from dominating the final decision. Experiments on the MATH benchmark show that MiniRep consistently outperforms conventional MAD aggregation and other reputation‑based approaches across a wide range of attack scenarios.
arXiv:2602. 09222v2 Announce Type: replace-cross Abstract: Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf.
arXiv:2609.00595v1 Announce Type: cross Abstract: Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating...
arXiv:2607. 06807v1 Announce Type: cross Abstract: While enabling effective collaboration on complex tasks, LLM-based Multi-Agent Systems (MAS) face critical security challenges due to vulnerabilities at the agent and interaction levels.
The paper introduces ReMAS, a reinforcement learning-based framework for black-box red-teaming of multi-agent systems (MAS) that fine-tunes attacker LLMs to craft adversarial prompts. ReMAS operates in two stages: a rewrite stage that refines base attack prompts to boost extraction success, and a template generation stage that builds attack templates to better invoke specific sub-agents and expose their system prompts. Experiments demonstrate that ReMAS significantly outperforms existing black-box attacks, showing high success rates and transferability across different backend LLMs and MAS structures.
arXiv:2606. 14517v1 Announce Type: cross Abstract: LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents.
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.