MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
By Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu
The paper introduces Many-Tier Instruction Hierarchy (ManyIH), a new framework for resolving conflicts among instructions with arbitrarily many privilege levels in large language model agents. It presents ManyIH-Bench, a benchmark featuring 853 agentic tasks that require navigating up to 12 levels of conflicting instructions across 46 real-world agents. Experiments show current models achieve only about 40% accuracy when instruction conflict scales, highlighting a gap in fine-grained, scalable conflict resolution.
By Jingyu Zhang, Tianjian Li, William Jurayj, Hongyuan Zhan, Benjamin Van Durme, Daniel Khashabi
The paper introduces AIM, a privacy‑aware memory framework that lets multi‑agent, multi‑user large language models manage both private and shared memory. AIM classifies data as private (user‑specific) or public (shared) and enforces index‑level access controls to protect sensitive information while enabling shared knowledge to improve coordination. The authors also present MUMBench, a new dataset for evaluating memory operations in multi‑user settings, and report high accuracy metrics for AIM on this benchmark.
By Zachary Johnson, Nigel Boachie Kumankumah, Somya Chatterjee, Tejas Sathyamurthi, Min Chen, Xinyi Alice Li, Xiao Wang, Emily Morgan Gelchie, Jessica Lin, Sadid A. Hasan, Sulaiman Vesal
arXiv:2607. 05477v1 Announce Type: cross Abstract: Improving the task performance of Large Language Models (LLMs) is essential, yet scaling these models faces significant challenges such as diminishing returns and high costs.
By Lars Benedikt Kaesberg
The paper "Cross-User Poisoning: User-Task Boundary Failures in Multi-User Collaborative Language Agents" examines how shared language agents, which handle interleaved messages from multiple users, can be exploited through cross-user poisoning (CUP). CUP occurs when an adversary injects a message into the shared context that later influences the agent’s actions for a benign user, leading to unauthorized responses or actions. The authors validate CUP on two deployed agents, Continua and ElizaOS, and introduce the MURMUR framework to evaluate such attacks. Their experiments across Slack, Workspace, and Airline domains show CUP’s high success rate and persistence, outperforming prompt-injection attacks. They also test defenses, finding that context summarization alone is insufficient, while task clustering and provenance prompting can significantly reduce non-adaptive propagation. The study concludes that robust multi-user agents must implement explicit user/task scoping rather than relying on generic input filtering or context compression.
By Atharv Singh Patlan, Peiyao Sheng, S. Ashwin Hebbar, Prateek Mittal, Pramod Viswanath
arXiv:2607. 25446v1 Announce Type: new Abstract: Multi-agent frameworks built on large language models (LLMs) routinely entangle three logically distinct concerns: who is on the team (organization), how members align (coordination), and which algorithm fuses their work (collaboration protocol).
By Huan Chen, Xiang Song, Jian Jin, Pan Ren, Liang-Jie Zhang
arXiv:2605. 09823v3 Announce Type: replace-cross Abstract: Personal AI assistants are beginning to act as delegates with access to calendars, inboxes, and user preferences.
By Chelsea Zou, Yiheng Yao, Selena She, Noah Goodman, Robert D. Hawkins
The paper introduces skilder, a framework that organizes LLM agent capabilities into role‑scoped bundles of skills, tools, and instructions, with explicit limits. Agents start with a minimal role catalog, discover the roles needed for a task, and receive the associated tools only through a single MCP server, ensuring deterministic enforcement of scope. Experiments on 13 tasks with six models show that skilder’s authorization layer prevents unauthorized tool calls and parameter violations while maintaining flexibility through dynamic cross‑role capability acquisition.
By Michael Stettler, Benjamin Girardet, Jonas Canton, Nicolas Corod
Multi-agent large language model (LLM) systems can expose protected state through internal messages, tool arguments, logs, and persistent memory even when their public outputs appear innocuous. Existing privacy prompts, redaction methods, and source-level access controls restrict surface content or data access, but do not specify what a legitimately informed agent should disclose or how that disclosure may be reused downstream.
PrivacyFlow introduces a new benchmark, PrivacyBench, and a defense system that separates privacy planning from task execution in autonomous LLM agents. The system traces data provenance, models recipient knowledge, and simulates inference attacks to weigh disclosure risk against task utility, allowing it to minimize or block data sharing. Experiments show that PrivacyFlow reduces privacy failures and increases completed task sequences by 22–55 percentage points, while revealing a tradeoff between privacy, utility, and user autonomy.
By Robin Staab, Tom Wollschl\"ager, Vahab Mirrokni, Reza Shokri
Structured Authority Injection (SAI) is a protocol that enables persistent human control over large language model (LLM) agents by routing operator corrections through typed, schema‑validated Model Context Protocol (MCP) tool calls and maintaining a ranked authority context in each prompt. It defines three override classes—Constraint, Priority, and Strategy—each with distinct persistence semantics and a strict precedence order enforced via a directed authority channel. Evaluations across nine LLMs and eight scenarios show reduced ambiguous directive interpretation, mitigated authority decay, and lower prompt overhead in the strongest models, though benefits vary by model.
By Ryan Tran, YueMing Zhang, Lingfeng Tao, Nan Yang, Shengjie Xu, Yu Yin, Shuai Xu, Wenlu Zhang, Hailu Xu
arXiv:2609.38816v1 Announce Type: new
Abstract: While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems r...
By Zongwan Cao, Ziyuan Yang, Shangbin Feng, Michael Duan, Skyler Hallinan, Bingbing Wen, Lucy Lu Wang, Yulia Tsvetkov