The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.
arXiv:2606. 26772v1 Announce Type: new Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training.
arXiv:2602. 06838v3 Announce Type: replace Abstract: Federated learning enables collaborative model training across distributed clients while preserving data privacy.
arXiv:2505. 22703v2 Announce Type: replace Abstract: Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.
arXiv:2407. 08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning.
arXiv:2607. 05866v1 Announce Type: cross Abstract: Under a fixed privacy budget, the utility of differentially private (DP) training is ultimately determined by its optimization efficiency.
arXiv:2607. 23649v1 Announce Type: new Abstract: Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction.
Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD). However, DPSGD suffers from substantial utility degradation and slow convergence due to gradient clipping and noise injection.
arXiv:2606. 04384v1 Announce Type: new Abstract: Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD).
arXiv:2510. 05416v3 Announce Type: replace Abstract: Differentially private stochastic gradient descent (DP-SGD) offers the promise of training deep learning models while mitigating many privacy risks.
arXiv:2601. 10237v3 Announce Type: replace Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is the dominant paradigm for private training, but its fundamental limitations under worst-case adversarial privacy definitions remain poorly understood.
arXiv:2607. 19580v1 Announce Type: new Abstract: Differentially private machine learning enables model training on sensitive data while ensuring that individual data is unlikely to be recoverable from the parameters of the resulting model.
arXiv:2606. 05435v1 Announce Type: new Abstract: Differentially private stochastic gradient descent (DP-SGD) has become the standard framework for privacy-preserving machine learning, yet its reliance on a fixed gradient clipping threshold to limit sensitivity remains a significant practical limitation.