arXiv Machine Learning October 19, 2026 By Tianze Wang, Zhiqi Bu, Linjun Zhang

Towards Hyperparameter Transfer for Differentially Private Optimization

Read the original on arXiv Machine Learning →

The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 26

Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

arXiv:2606. 26772v1 Announce Type: new Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training.

By Naoki Nishikawa, Shokichi Takakura, Satoshi Hasegawa

rag diffusion fine-tuning safety

arXiv AI

Jul 9

An Adaptive Differentially Private Federated Learning Framework

arXiv:2602. 06838v3 Announce Type: replace Abstract: Federated learning enables collaborative model training across distributed clients while preserving data privacy.

By Jin Wang, Hui Ma, Yajun Zhang, Xinjun Pei, Ming Yan, Fei Xing, Yikun Chen

safety

arXiv Machine Learning

Jul 1

Private Rate-Constrained Optimization with Applications to Fair Learning

arXiv:2505. 22703v2 Announce Type: replace Abstract: Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.

By Mohammad Yaghini, Tudor Cebere, Michael Menart, Aur\'elien Bellet, Nicolas Papernot

safety

arXiv Machine Learning

Jul 23

Differentially Private Neural Network Training Under the Hidden State Assumption

arXiv:2407. 08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning.

By Ding Chen, Haochen Luo, Xiaofei Wang, Chen Liu

benchmarks safety

arXiv AI

Jul 8

Differentially Private Natural Gradient Descent

arXiv:2607. 05866v1 Announce Type: cross Abstract: Under a fixed privacy budget, the utility of differentially private (DP) training is ultimately determined by its optimization efficiency.

By Pan Li, Kai Chen, Shuai Chang, Shengzhi Zhang, Peizhuo Lv, Jinwen He

benchmarks safety

arXiv Machine Learning

Jul 28

DP-IVON-Gradsq: Differentially Private Squared-Gradient Improved Variational Online Newton

arXiv:2607. 23649v1 Announce Type: new Abstract: Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction.

By Nour Jamoussi, Ikram Dridi, Giuseppe Serra, Marios Kountouris

safety

One email a morning, machine-written

Related stories

Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

An Adaptive Differentially Private Federated Learning Framework

Private Rate-Constrained Optimization with Applications to Fair Learning

Differentially Private Neural Network Training Under the Hidden State Assumption

Differentially Private Natural Gradient Descent

DP-IVON-Gradsq: Differentially Private Squared-Gradient Improved Variational Online Newton