arXiv Computation and Language
Multi-User Large Language Model Agents
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
1d ago
MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability
MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
arXiv AI
Sep 10
Many-Tier Instruction Hierarchy in LLM Agents
The paper introduces Many-Tier Instruction Hierarchy (ManyIH), a new framework for resolving conflicts among instructions with arbitrarily many privilege levels in large language model agents. It presents ManyIH-Bench, a benchmark featuring 853 agentic tasks that require navigating up to 12 levels of conflicting instructions across 46 real-world agents. Experiments show current models achieve only about 40% accuracy when instruction conflict scales, highlighting a gap in fine-grained, scalable conflict resolution.
arXiv Machine Learning
Sep 14
AIM: A Privacy-Aware Interoperable Memory Framework for Multi-Agent Multi-User LLM Systems
The paper introduces AIM, a privacy‑aware memory framework that lets multi‑agent, multi‑user large language models manage both private and shared memory. AIM classifies data as private (user‑specific) or public (shared) and enforces index‑level access controls to protect sensitive information while enabling shared knowledge to improve coordination. The authors also present MUMBench, a new dataset for evaluating memory operations in multi‑user settings, and report high accuracy metrics for AIM on this benchmark.
arXiv AI
Jul 8
Decision Protocols in Multi-Agent Large Language Model Conversations
arXiv:2607. 05477v1 Announce Type: cross Abstract: Improving the task performance of Large Language Models (LLMs) is essential, yet scaling these models faces significant challenges such as diminishing returns and high costs.
arXiv AI
Oct 28
Cross-User Poisoning: User-Task Boundary Failures in Multi-User Collaborative Language Agents
The paper "Cross-User Poisoning: User-Task Boundary Failures in Multi-User Collaborative Language Agents" examines how shared language agents, which handle interleaved messages from multiple users, can be exploited through cross-user poisoning (CUP). CUP occurs when an adversary injects a message into the shared context that later influences the agent’s actions for a benign user, leading to unauthorized responses or actions. The authors validate CUP on two deployed agents, Continua and ElizaOS, and introduce the MURMUR framework to evaluate such attacks. Their experiments across Slack, Workspace, and Airline domains show CUP’s high success rate and persistence, outperforming prompt-injection attacks. They also test defenses, finding that context summarization alone is insufficient, while task clustering and provenance prompting can significantly reduce non-adaptive propagation. The study concludes that robust multi-user agents must implement explicit user/task scoping rather than relying on generic input filtering or context compression.
arXiv AI
Jul 29
Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
arXiv:2607. 25446v1 Announce Type: new Abstract: Multi-agent frameworks built on large language models (LLMs) routinely entangle three logically distinct concerns: who is on the team (organization), how members align (coordination), and which algorithm fuses their work (collaboration protocol).