arXiv Machine Learning

Differentially Private Synthetic Data via Generalized Moment Matching

arXiv:2610. 30196v1 Announce Type: cross Abstract: We study $(\varepsilon,\delta)$-differentially private synthetic data for arbitrary databases of $n$ records in $[-1,1]^d$ under a specified integral probability metric (IPM).

arXiv Machine Learning
5d ago

Differentially Private Synthetic Data from Sliced Chebyshev Moments

The paper introduces SPLICE, a differentially private synthetic data algorithm that uses noisy sliced Chebyshev polynomial moments to fit a distribution. SPLICE achieves a Sliced Wasserstein-1 distance of ∼O(1/√n) from the target distribution, a bound that does not depend on the data dimension. The authors also prove a new moment‑matching result and provide a practical implementation that outperforms existing DP synthetic data methods on a large benchmark.

By Lucas Rosenblatt, R. Teal Witter, Christopher Musco
arXiv Machine Learning
Oct 28

Gaussian Approximation for Composition and Subsampling of Differentially Private Algorithms

The paper presents a Gaussian approximation for the privacy loss distribution in differentially private stochastic gradient descent (DP‑SGD) when subsampling is used. It shows that, after truncating rare large losses, the privacy loss behaves like a Gaussian with the same mean and variance, up to a small shift that grows only logarithmically with the error tolerance. This leads to tighter variance‑based bounds for subsampled mechanisms, improving upon previous Renyi‑divergence analyses and explaining the effectiveness of the connect‑the‑dots method for random allocation sampling.

By Vitaly Feldman, Moshe Shenfeld
arXiv Machine Learning
Oct 12

Locally Private Parametric Methods for Change-Point Detection

The paper investigates parametric change‑point detection under local differential privacy, concentrating on offline localization of a single change point in independent observations. It derives improved finite‑sample accuracy guarantees for a generalized likelihood‑ratio estimator using martingale methods and establishes finite‑sample minimax lower bounds in both non‑private and locally private settings, linking localization accuracy to Chernoff information. Guided by these bounds, the authors design locally private algorithms based on aggregated randomized response and binary mechanisms, optimize Chernoff information in different privacy regimes, and provide finite‑sample guarantees, while also computing strong data‑processing inequality coefficients for certain divergences and validating the results empirically on synthetic data.

By Anuj Kumar Yadav, Cemre Cadir, Yanina Y. Shkel, Michael Gastpar
arXiv Machine Learning
1d ago

Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics

The paper analyzes deep nonparametric regression trained with stochastic gradient Langevin dynamics (SGLD), a stochastic gradient method that adds Gaussian noise. Using a PAC‑Bayesian framework for deep ReLU networks, it derives an excess‑risk bound that incorporates network approximation error, statistical complexity, and the cumulative effect of the SGLD trajectory. The study also provides differential privacy guarantees for per‑sample clipped SGLD and demonstrates the method’s predictive performance and privacy‑utility trade‑offs through simulations and real data experiments.

By Yeheng Ge, Jian Huang, Shuyuan Wu