arXiv Machine Learning

Exploring the Privacy--Size Tradeoff for Differentially Private Self-Attentive Variational Autoencoders

Hugging Face Trending Papers
Oct 28

Fed-IGTD: A federated learning-based framework for tabular-to-image encoding

Fed-IGTD is a federated learning framework that enables tabular-to-image encoding without centralizing data. It reconstructs a global correlation matrix from local statistics and trains a CNN via Federated Averaging, while applying two differential privacy mechanisms: a one-shot Gaussian noise addition during correlation aggregation and a client‑level DP‑FedAvg during training. Experiments on five datasets show that privacy protection during mapping construction has minimal impact on performance, whereas DP‑FedAvg can significantly degrade utility, especially with small client cohorts and moderate privacy budgets.

arXiv Machine Learning
Oct 19

Towards Hyperparameter Transfer for Differentially Private Optimization

The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.

By Tianze Wang, Zhiqi Bu, Linjun Zhang
arXiv Machine Learning
Jun 26

Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

arXiv:2606. 26772v1 Announce Type: new Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training.

By Naoki Nishikawa, Shokichi Takakura, Satoshi Hasegawa
arXiv Machine Learning
Oct 8

Efficient Provably Private Classification with a Tabular Foundation Model

Efficient Provably Private Classification with a Tabular Foundation Model introduces PrivTab, a tabular foundation model that embeds differential privacy directly into its architecture. Pretrained on simulated datasets, PrivTab uses in-context learning to convert sensitive rows into compact, provably private summaries, enabling a single forward pass for classification. The model outperforms private linear and neural‑network baselines under moderate-to-strong privacy, shows negligible membership leakage, maintains well‑calibrated predictions, and reduces dataset fitting time by 10,000×.

By Talal Alrawajfeh, Cristiana Diaconu, Ossi R\"ais\"a, Sebastian Rodriguez Beltran, Yuan He, John Bronskill, Richard E. Turner, Antti Honkela
arXiv Machine Learning
Jul 23

Differentially Private Neural Network Training Under the Hidden State Assumption

arXiv:2407. 08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning.

By Ding Chen, Haochen Luo, Xiaofei Wang, Chen Liu