arXiv Machine Learning

CALLIOPE: Private Insights into AI Use, Without Embeddings

CALLIOPE is a new top‑down framework designed to uncover topic hierarchies in large unstructured datasets, such as LLM conversations, while ensuring differential privacy. Unlike conventional methods that depend on clustering vector embeddings, CALLIOPE directly employs LLM capabilities throughout its pipeline. Experiments show that CALLIOPE delivers higher utility than embedding‑based approaches, particularly in high‑privacy scenarios, and its outputs are evaluated using both standard clustering metrics and newly introduced utility metrics.

arXiv Machine Learning

Oct 16

SHIP: Structured and Hierarchical Insight with Privacy

The paper introduces SHIP, a method that generates differentially private insights from textual data by mapping queries to attributes, constructing a hierarchical hybrid taxonomy, and releasing a DP synthetic table for analysis. It treats these insights as reusable representations to answer analytical queries, enabling a common basis for utility assessment. Experiments show SHIP outperforms existing private insight baselines in QA performance and structured fidelity, while highlighting remaining challenges for structured representations.

By Kai Chen, Guisi Ni, Albert Cheu, Kassem Fawaz, Yuzheng Hu, Ryan McKenna, Peter Kairouz

safety

arXiv Machine Learning

Sep 16

Differentially Private Semantic Plans for Aggregate Insight Generation

The paper introduces ‘DP-SPIN’, a trusted‑curator framework that generates differentially private semantic plans for aggregate insight generation. ‘DP-SPIN’ maps each record to a bounded sparse nonnegative vector over pre‑defined semantic concepts, sums these vectors into a semantic sketch, and releases a noisy plan containing admitted concepts and their masses. The framework provides user‑level privacy by clipping each user’s contribution and ensures that the final summary is differentially private through post‑processing, with guarantees established under both add/drop and replacement adjacency.

By Behrooz Razeghi

llms nlp safety

arXiv AI

Oct 16

Private Retrieval Augmented Generation via Random Projection

The paper introduces a lightweight privacy-preserving technique for Retrieval-Augmented Generation (RAG) systems that uses random projection to transform both datastore and query embeddings. This approach maintains semantic similarity for effective retrieval while significantly reducing the risk of data extraction attacks. Experiments across various RAG architectures and datasets demonstrate that the method outperforms prior differential privacy-based techniques in both retrieval and generation performance while offering comparable empirical privacy.

By Dixi Yao, Tahseen Rabbani, Tian Li

llms rag safety

arXiv Machine Learning

3d ago

Guided Diffusion for Trustworthy Synthetic Text Generation

arXiv:2610.27259v1 Announce Type: cross Abstract: The rapid evolution of Large Language Models is fueled by massive datasets, yet this reliance poses severe privacy risks, as models can inadvertently...

By Qi Pang, Hanshen Xiao, Andre Manoel, Lipika Ramaswamy, Wenting Zheng, Virginia Smith, G. Edward Suh

llms rag diffusion fine-tuning safety

arXiv Machine Learning

Sep 1

Revisiting the Provable-Auditable Privacy Gap of DP-SGD

arXiv:2608.28934v1 Announce Type: new Abstract: Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In...

By Saloni Modi, Srivi Balaji, Yusong Zhu, Gautam Kamath, Kevin Tian

benchmarks safety

arXiv AI

Jul 22

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

arXiv:2606. 16952v2 Announce Type: replace-cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.

By Kareem Amin, Rudrajit Das, Alessandro Epasto, Adel Javanmard, Dennis Kraft, M\'onica Ribero, Sergei Vassilvitskii

llms safety

arXiv AI

Jul 28

PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window

arXiv:2607. 22695v1 Announce Type: new Abstract: Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks, leading to widespread deployment.

By Ryan Thornton, Mir Mehedi Ahsan Pritom, Maanak Gupta

llms benchmarks safety

arXiv Machine Learning

Jul 23

Differentially Private Neural Network Training Under the Hidden State Assumption

arXiv:2407. 08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning.

By Ding Chen, Haochen Luo, Xiaofei Wang, Chen Liu

benchmarks safety

arXiv Computation and Language

Sep 22

LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication

arXiv:2609.23193v1 Announce Type: cross Abstract: As Large Language Model (LLM) APIs become increasingly integrated into privacy-sensitive workflows, ensuring inference-time privacy without compromis...

By Yuzhu Mao, Liang Zhao

llms safety

arXiv AI

Aug 5

PRIVEE: Privacy-Preserving Vertical Federated Learning Against Feature Inference Attacks

arXiv:2512. 12840v2 Announce Type: replace-cross Abstract: Vertical Federated Learning (VFL) enables collaborative model training across organizations that share common user samples but hold disjoint feature spaces.

By Sindhuja Madabushi, Haider Ali, Ahmad Faraz Khan, Rui Ning, Hongyi Wu, Chunsheng Xin, Ali. R. Butt, Jin-Hee Cho

safety

arXiv AI

Jul 2

NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents

arXiv:2601. 14660v2 Announce Type: replace-cross Abstract: Agentic Large Language Models (LLMs) are models able to reason, plan, and execute tools over unstructured data.

By Saswat Das, Ferdinando Fioretto

llms agents safety

arXiv Machine Learning

Jun 24

Natural Identifiers for Privacy and Data Audits in Large Language Models

arXiv:2606. 24408v1 Announce Type: new Abstract: Assessing the privacy of large language models (LLMs) presents significant challenges.

By Lorenzo Rossi, Bart{\l}omiej Marek, Franziska Boenisch, Adam Dziedzic

llms safety

Related stories