arXiv Machine Learning

Noise-Aware Differentially Private Variational Inference

arXiv Machine Learning
Oct 2

Certification-Based Differentially Private Learning

The paper extends the abstract gradient training (AGT) framework to provide tighter differential privacy guarantees for both private prediction and private learning. It introduces Abstract Gradient Sampling (AGS) to analyze privacy in continuous, unbounded regression and offers theoretical and empirical evidence that these methods yield tighter bounds than global-sensitivity baselines, even in previously unbounded settings. The authors also demonstrate that their private learning algorithm can outperform standard private learners under comparable conditions.

By Mihnea Ghitu, Matthew Wicker
arXiv Machine Learning
1d ago

Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics

The paper analyzes deep nonparametric regression trained with stochastic gradient Langevin dynamics (SGLD), a stochastic gradient method that adds Gaussian noise. Using a PAC‑Bayesian framework for deep ReLU networks, it derives an excess‑risk bound that incorporates network approximation error, statistical complexity, and the cumulative effect of the SGLD trajectory. The study also provides differential privacy guarantees for per‑sample clipped SGLD and demonstrates the method’s predictive performance and privacy‑utility trade‑offs through simulations and real data experiments.

By Yeheng Ge, Jian Huang, Shuyuan Wu
arXiv Machine Learning
Oct 16

Differentially Private Distribution Release of Gaussian Mixture Models via KL-Divergence Minimization

The paper proposes a method for releasing Gaussian Mixture Models (GMMs) under differential privacy (DP) by minimizing Kullback‑Leibler (KL) divergence between the released and non‑private models. It jointly optimizes privacy‑budget allocation and parameter perturbations for mixture weights, means, and covariances, providing a tractable expression for KL divergence and a theoretical analysis of DP guarantees. Experiments on synthetic and real data show that the approach achieves strong privacy while preserving high utility.

By Hang Liu, Anna Scaglione, Sean Peisert