arXiv Machine Learning
Oct 8
Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
arXiv:2610. 09651v1 Announce Type: new Abstract: DP-SGD protects training data by adding Gaussian noise to clipped gradients.
Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
Related stories
arXiv Machine Learning
Oct 5
Trade-off Functions for DP-SGD with Subsampling based on Random Allocation: Tight Upper and Lower Bounds
arXiv:2605. 06259v3 Announce Type: replace Abstract: Within the $f$-DP framework, we derive a tight analysis of the trade-off function for Differentially Private Stochastic Gradient Descent (DP-SGD) with subsampling based on random allocation in which each sample is independently assigned to exactly one of $M$ minibatches per epoch, each minibatch corresponding to one of the $M$ SGD rounds within a single epoch.
arXiv Machine Learning
Aug 11
Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD
arXiv:2601. 10237v3 Announce Type: replace Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is the dominant paradigm for private training, but its fundamental limitations under worst-case adversarial privacy definitions remain poorly understood.
arXiv Machine Learning
Oct 9
The Sample Complexity of Membership Inference and Privacy Auditing
The paper investigates the sample complexity required for membership‑inference attacks, focusing on Gaussian mean estimation. It shows that, in this setting, an attacker may need at least Ω(n + n^2 ho^2) reference samples to match a fully informed adversary, which can be far more than the n samples used by the learning algorithm. This demonstrates that practical attacks, which typically use only O(n) samples, might underestimate the true risk of membership inference.
arXiv Machine Learning
Sep 14
Membership Inference via Pairwise Likelihood Ratios
The paper introduces Pairwise Likelihood MIA (PL‑MIA), a unified membership inference attack that combines a Gaussian likelihood‑ratio statistic with population calibration and the Cauchy combination test. PL‑MIA generates p‑values from pairwise comparisons between a query point and reference points, then aggregates these continuous signals using the Cauchy test to preserve evidence strength. Experiments show that PL‑MIA surpasses strong baselines, boosting true positive rates by over 25% in low‑false‑positive settings, thereby validating the theoretical advantages of the proposed statistical framework.
arXiv Machine Learning
Aug 11
Score-based Membership Inference on Diffusion Models
arXiv:2509. 25003v3 Announce Type: replace Abstract: Membership inference attacks (MIAs) against Diffusion Models (DMs) raise pressing privacy concerns by revealing whether a sample was part of the training set.
arXiv Machine Learning
Sep 18
Near-Optimal Machine Unlearning Utility for Smooth Strongly Convex Losses
arXiv:2606. 01527v2 Announce Type: replace Abstract: Machine unlearning is motivated by legal and user-facing requirements to remove the influence of individuals' data from trained models, such as the right to be forgotten.
arXiv Machine Learning
Oct 9
Minimax Gaussian Mechanisms for Continual Machine Unlearning
The paper introduces Gaussian mechanisms for Newton updates that enable continual machine unlearning—updating a model after record deletions without full retraining. Using Gaussian differential privacy and adaptive composition, it shows that the sequence of released models is statistically indistinguishable from exact retraining. The authors derive error bounds for the Newton approximation, calibrate independent and random‑walk Gaussian noise, and demonstrate minimax optimality under various residual and increment bounds, with empirical validation on simulations and credit‑default data.
arXiv Machine Learning
Jun 15
Private Prediction via PAC Privacy
arXiv:2601. 14033v2 Announce Type: replace Abstract: Machine learning models are increasingly served behind APIs.
arXiv Machine Learning
Sep 22
Improved Private Sparse Covariance Estimation with Multiscale Threshold Tests
arXiv:2609. 22783v1 Announce Type: new Abstract: We study differentially private covariance estimation in operator norm for mean-zero sub-Gaussian distributions with unknown covariance support and at most $k$ nonzero entries per row.
arXiv Machine Learning
5d ago
Gaussian Approximation for Composition and Subsampling of Differentially Private Algorithms
The paper presents a Gaussian approximation for the privacy loss distribution in differentially private stochastic gradient descent (DP‑SGD) when subsampling is used. It shows that, after truncating rare large losses, the privacy loss behaves like a Gaussian with the same mean and variance, up to a small shift that grows only logarithmically with the error tolerance. This leads to tighter variance‑based bounds for subsampled mechanisms, improving upon previous Renyi‑divergence analyses and explaining the effectiveness of the connect‑the‑dots method for random allocation sampling.
arXiv AI
Sep 10
PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement
The paper introduces PAC‑Private Autoregressive Generation, a method that calibrates noise based on ensemble disagreement across overlapping ‘worlds’ of a private corpus, thereby extending PAC privacy from classification to text generation. By training adapters on a frozen public model and using posterior‑weighted disagreement to add noise only when predictions vary, the approach achieves strong privacy guarantees while preserving most of the fine‑tuning benefit. Experiments on WikiText‑103 with GPT‑2‑small show 74 % of the fine‑tuning gain retained with a per‑token budget of 2⁻³², and membership‑inference success bounded to 51.08 % after one million tokens, outperforming PMixED under matched conditions.