arXiv Machine Learning

On Inherent Privacy of Posterior Sampling: A Unified R\'enyi-Divergence Framework

arXiv:2610. 19573v1 Announce Type: cross Abstract: Differential Privacy (DP) provides a rigorous framework for quantifying privacy guarantees.

arXiv Statistics ML
Oct 7

One-Shot Private Confidence Regions via Resampling

arXiv:2610.08460v1 Announce Type: new Abstract: We propose a simple framework for constructing differentially private confidence regions \textit{in one shot}, i.e., by adding noise only to the final...

By Shourya Pandey, Purnamrita Sarkar, Po-Ling Loh, Debepsita Mukherjee
arXiv Machine Learning
Oct 28

Gaussian Approximation for Composition and Subsampling of Differentially Private Algorithms

The paper presents a Gaussian approximation for the privacy loss distribution in differentially private stochastic gradient descent (DP‑SGD) when subsampling is used. It shows that, after truncating rare large losses, the privacy loss behaves like a Gaussian with the same mean and variance, up to a small shift that grows only logarithmically with the error tolerance. This leads to tighter variance‑based bounds for subsampled mechanisms, improving upon previous Renyi‑divergence analyses and explaining the effectiveness of the connect‑the‑dots method for random allocation sampling.

By Vitaly Feldman, Moshe Shenfeld
arXiv Machine Learning
Oct 16

Differentially Private Distribution Release of Gaussian Mixture Models via KL-Divergence Minimization

The paper proposes a method for releasing Gaussian Mixture Models (GMMs) under differential privacy (DP) by minimizing Kullback‑Leibler (KL) divergence between the released and non‑private models. It jointly optimizes privacy‑budget allocation and parameter perturbations for mixture weights, means, and covariances, providing a tractable expression for KL divergence and a theoretical analysis of DP guarantees. Experiments on synthetic and real data show that the approach achieves strong privacy while preserving high utility.

By Hang Liu, Anna Scaglione, Sean Peisert
arXiv Machine Learning
1d ago

Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics

The paper analyzes deep nonparametric regression trained with stochastic gradient Langevin dynamics (SGLD), a stochastic gradient method that adds Gaussian noise. Using a PAC‑Bayesian framework for deep ReLU networks, it derives an excess‑risk bound that incorporates network approximation error, statistical complexity, and the cumulative effect of the SGLD trajectory. The study also provides differential privacy guarantees for per‑sample clipped SGLD and demonstrates the method’s predictive performance and privacy‑utility trade‑offs through simulations and real data experiments.

By Yeheng Ge, Jian Huang, Shuyuan Wu