The paper analyzes deep nonparametric regression trained with stochastic gradient Langevin dynamics (SGLD), a stochastic gradient method that adds Gaussian noise. Using a PAC‑Bayesian framework for deep ReLU networks, it derives an excess‑risk bound that incorporates network approximation error, statistical complexity, and the cumulative effect of the SGLD trajectory. The study also provides differential privacy guarantees for per‑sample clipped SGLD and demonstrates the method’s predictive performance and privacy‑utility trade‑offs through simulations and real data experiments.
arXiv:2607. 23649v1 Announce Type: new Abstract: Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction.
arXiv:2510. 05416v3 Announce Type: replace Abstract: Differentially private stochastic gradient descent (DP-SGD) offers the promise of training deep learning models while mitigating many privacy risks.
arXiv:2407. 04884v4 Announce Type: replace Abstract: The hidden state threat model of differential privacy (DP) assumes that the adversary has access only to the final trained machine learning (ML) model, without seeing intermediate states during training.
arXiv:2601. 10237v3 Announce Type: replace Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is the dominant paradigm for private training, but its fundamental limitations under worst-case adversarial privacy definitions remain poorly understood.
arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.
arXiv:2505. 22703v2 Announce Type: replace Abstract: Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.
arXiv:2605. 06259v3 Announce Type: replace Abstract: Within the $f$-DP framework, we derive a tight analysis of the trade-off function for Differentially Private Stochastic Gradient Descent (DP-SGD) with subsampling based on random allocation in which each sample is independently assigned to exactly one of $M$ minibatches per epoch, each minibatch corresponding to one of the $M$ SGD rounds within a single epoch.
arXiv:2606. 01527v2 Announce Type: replace Abstract: Machine unlearning is motivated by legal and user-facing requirements to remove the influence of individuals' data from trained models, such as the right to be forgotten.
The paper provides the first optimization risk bounds for two‑layer Kolmogorov‑Arnold Networks (KANs) trained with clipped mini‑batch differentially private stochastic gradient descent (DP‑SGD) that uses temporally correlated noise. The bounds explicitly capture the effects of temporal correlation, clipping, mini‑batch sampling, and network width, and show that correlation can reduce leading noise terms while the clipping threshold influences an effective step size. Experiments on synthetic data and MNIST confirm the theoretical predictions, and the authors extend the results to population risk guarantees via algorithmic stability, recovering several known special cases.
arXiv:2610. 12865v1 Announce Type: new Abstract: Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly bounded noise variance.
The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.
arXiv:2607. 05866v1 Announce Type: cross Abstract: Under a fixed privacy budget, the utility of differentially private (DP) training is ultimately determined by its optimization efficiency.