arXiv AI September 2, 2026 By Krithika Ramesh, Krishna Pillutla, Danish Pruthi, Anjalie Field

The Privacy-Hallucination Tradeoff in Differentially Private Language Models

The paper investigates a privacy‑hallucination tradeoff in differentially private (DP) language models. It shows that DP pre‑training or fine‑tuning increases hallucinations compared to non‑DP models, especially as the privacy budget becomes stricter. The authors attribute this to DP mechanisms flattening output distributions, and demonstrate that controlling the frequency of facts in training data can mitigate hallucination risks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 1

Revisiting the Provable-Auditable Privacy Gap of DP-SGD

arXiv:2608.28934v1 Announce Type: new Abstract: Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In...

By Saloni Modi, Srivi Balaji, Yusong Zhu, Gautam Kamath, Kevin Tian

benchmarks safety

arXiv AI

Jul 22

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

arXiv:2606. 16952v2 Announce Type: replace-cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.

By Kareem Amin, Rudrajit Das, Alessandro Epasto, Adel Javanmard, Dennis Kraft, M\'onica Ribero, Sergei Vassilvitskii

llms safety

arXiv AI

Sep 4

VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models

The paper introduces VoxPrivacy, a benchmark for assessing interactional privacy in Speech Language Models (SLMs). It evaluates models on a 32‑hour bilingual dataset across three difficulty tiers, revealing that most open‑source SLMs perform near random on conditional privacy decisions and even strong closed‑source systems struggle with proactive privacy inference. The authors also validate these findings on a real‑speech subset and show that fine‑tuning on a 4,000‑hour training set can improve privacy‑preserving capabilities while maintaining robustness.

By Yuxiang Wang, Hongyu Liu, Dekun Chen, Xueyao Zhang, Zhizheng Wu

llms fine-tuning benchmarks safety

arXiv Machine Learning

Aug 31

Performative Privacy: When Differential Privacy Maximizes Utility

arXiv:2608.28198v1 Announce Type: new Abstract: Privacy-preserving learning is often motivated by the idea that protecting users' data can preserve trust and thus participation, improving utility in...

By Uddalak Mukherjee, Edwige Cyffers, Yann Chevaleyre

agents safety

arXiv AI

Jun 16

Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data

arXiv:2606. 16952v1 Announce Type: cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.

By Kareem Amin, Rudrajit Das, Alessandro Epasto, Adel Javanmard, Dennis Kraft, M\'onica Ribero, Sergei Vassilvitskii

llms safety

arXiv Machine Learning

Jun 9

Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models

arXiv:2606. 09401v1 Announce Type: new Abstract: Recent work has applied differential privacy (DP) to adapt large language models (LLMs) for sensitive applications, offering theoretical guarantees.

By Bart{\l}omiej Marek, Lorenzo Rossi, Vincent Hanke, Xun Wang, Michael Backes, Franziska Boenisch, Adam Dziedzic

llms fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

Revisiting the Provable-Auditable Privacy Gap of DP-SGD

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models

Performative Privacy: When Differential Privacy Maximizes Utility

Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data

Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models