arXiv Computation and Language

Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems

The paper investigates how large language model agents can inadvertently leak confidential information by combining data from multiple documents—a phenomenon termed the "mosaic effect." Using 30 synthetic cybersecurity, medical, and military scenarios, the authors generate 10,800 simulated and 1,440 live answers from three LLM agents, finding that even the best model discloses forbidden composites in over 7% of simulated responses. While visible classification headers and aggregation rules reduce disclosure rates, unsafe disclosures remain in roughly one third of simulated answers, and increased refusals do not fully prevent leakage of prohibited conclusions.

arXiv AI

Oct 27

InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants

InferPriv is a new benchmark that evaluates inferential privacy leakage in large language model (LLM) assistants. It measures whether an honest-but-curious recipient can deduce a user’s sensitive attributes from an LLM’s reply during routine tasks, a type of leakage not captured by existing direct‑leakage benchmarks. Testing five frontier models, the study finds that inferential leakage rates (16.2–25.2%) far exceed direct leakage, even when direct leakage is near zero under privacy‑aware prompts.

By Pengrun Huang, Yiwei Fu, Luoxi Meng, Kamalika Chaudhuri

llms agents benchmarks safety

arXiv AI

Sep 15

Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control

arXiv:2609.14003v1 Announce Type: cross Abstract: Personal AI agents built on large language models (LLMs) are increasingly given access to a user's private data and communications in order to provid...

By Minsun Shim, Ramisha Raida Karim, Ruthwik Jakkula, Kaiwen Zhou, Xin Liu, Xin Eric Wang, Zhou Li

llms agents benchmarks safety

arXiv AI

Jun 2

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

arXiv:2606. 00152v1 Announce Type: cross Abstract: LLM-based agents are rapidly advancing, autonomously invoking external tools to complete multi-step tasks for users.

By Mingxuan Zhang, Jiahui Han, Dadi Guo, Songze Li, Guanchu Wang, Na Zou, Dongrui Liu, Xia Hu

llms agents benchmarks safety

arXiv AI

Jun 17

An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios

arXiv:2606. 17114v1 Announce Type: cross Abstract: AI agents are increasingly being adopted in enterprise and personal settings with access to emails, databases, documents, and other tools where they can read, update, and disseminate sensitive information.

By Hankyul Baek, Jaewon Noh, Sang Seo, Yongsu Kim, Gabriel Waikin Loh Matienzo, Young Il Kim, Ee Wei Seah, Akriti Vij

llms agents safety

arXiv AI

Jun 26

Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents

arXiv:2606. 26627v1 Announce Type: cross Abstract: Large language model agents increasingly query databases, search document collections, call external APIs, remember past interactions, and act on a user's behalf.

By Nada Lahjouji, Ashwin Gerard Colaco

llms rag agents benchmarks safety data-engineering

arXiv AI

Jun 10

The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans

arXiv:2606. 09844v1 Announce Type: cross Abstract: Large Language Models (LLMs) alter their privacy behavior based on the perceived identity of their interlocutor.

By Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah

llms agents benchmarks safety

arXiv Machine Learning

Sep 3

Context Inference Attacks Without Jailbreaks

The paper investigates privacy risks in agentic AI systems that assemble sensitive data into a hidden context before responding. It introduces context‑inference attacks, a security game that evaluates how well attackers can recover this hidden context under varying levels of knowledge and indirect delivery. Experiments show that even with controls such as instructions not to disclose, logit suppression, and context dilution, agents can leak significant contextual information, achieving high success rates across multiple attack settings.

By Prince Jha, Samuele Poppi, Nils Lukas

agents multimodal safety

arXiv AI

Sep 21

CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents

CIPL (Channel Inversion for Privacy Leakage) is a channel-aware framework designed to evaluate black-box privacy leakage in large language model agents. It models the leakage process through stages of sensitive source, selection, assembly, execution, observation, and extraction, assessing how selected sensitive units become attacker-recoverable outputs. Experiments across memory, retrieval, and tool-mediated targets, plus a live-agent case study, reveal that recoverability depends on factors beyond storage labels, such as observation surface, prompt alignment, retrieval depth, and provider behavior, and that a semantic audit can uncover disclosures missed by exact matching.

By Tao Huang, Guosen Wu, Guolong Zheng, Jiayang Meng, Chen Hou, Xu Yang, Xuechao Yang, Feng Xia

llms agents safety

Hugging Face Trending Papers

Aug 3

MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication

Multi-agent large language model (LLM) systems can expose protected state through internal messages, tool arguments, logs, and persistent memory even when their public outputs appear innocuous. Existing privacy prompts, redaction methods, and source-level access controls restrict surface content or data access, but do not specify what a legitimately informed agent should disclose or how that disclosure may be reused downstream.

llms agents safety

arXiv Computation and Language

Oct 27

"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents

arXiv:2606.00497v2 Announce Type: replace-cross Abstract: Deceptive web content, widely instantiated across the internet and commonly known as \textit{social-engineering attacks}, manipulates autonom...

By Soham Roy, Sarthakbrata Halder, Arya Bharaty, Vaibhav Bhaskar, Yash Sinha, Dhruv Kumar, Srikant Panda, Murari Mandal

llms agents benchmarks safety

arXiv AI

Jun 16

AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems

arXiv:2602. 11510v3 Announce Type: replace Abstract: Multi-agent Large Language Model (LLM) systems create privacy risks that current output-only benchmarks cannot measure.

By Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah

llms agents benchmarks safety

arXiv AI

Jun 2

Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation

arXiv:2512. 16310v3 Announce Type: replace-cross Abstract: LLM-based agents increasingly use multiple external tools to complete complex tasks.

By Yuxuan Qiao, Dongqin Liu, Hongchang Yang, Wei Zhou, Songlin Hu

llms agents reinforcement-learning fine-tuning benchmarks safety

Related stories