The paper investigates high‑dimensional LASSO under differential privacy using objective perturbation when covariates have heterogeneous scales. It introduces a Gram‑based anisotropic objective perturbation that counteracts the distortion caused by covariate heterogeneity, restoring isotropy in the estimation process. Through an Approximate Message Passing framework and state evolution analysis, the authors show that this approach stabilizes convergence and improves both statistical efficiency and privacy performance compared to standard uniform noise injection.
arXiv:2607. 05866v1 Announce Type: cross Abstract: Under a fixed privacy budget, the utility of differentially private (DP) training is ultimately determined by its optimization efficiency.
arXiv:2508.04800v2 Announce Type: replace-cross Abstract: We introduce a novel privatization framework for high-dimensional controlled variable selection. Our framework enables rigorous False Discove...
The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.
arXiv:2602. 06838v3 Announce Type: replace Abstract: Federated learning enables collaborative model training across distributed clients while preserving data privacy.
arXiv:2606. 18773v1 Announce Type: cross Abstract: We study differentially private (DP) regression in settings where each data sample includes public, non-sensitive features -- common in applications such as recommendation and advertising systems.
The paper proposes a method for releasing Gaussian Mixture Models (GMMs) under differential privacy (DP) by minimizing Kullback‑Leibler (KL) divergence between the released and non‑private models. It jointly optimizes privacy‑budget allocation and parameter perturbations for mixture weights, means, and covariances, providing a tractable expression for KL divergence and a theoretical analysis of DP guarantees. Experiments on synthetic and real data show that the approach achieves strong privacy while preserving high utility.
arXiv:2505. 22703v2 Announce Type: replace Abstract: Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.
arXiv:2303. 07152v3 Announce Type: replace-cross Abstract: Achieving optimal statistical performance while ensuring the privacy of personal data is a challenging yet crucial objective in modern data analysis.
The paper introduces Jacobian-Guided Anisotropic Noise Reshaping, a method that improves data utility under Local Differential Privacy by selectively reducing noise in task-relevant subspaces of data representations. It uses the Jacobian of a public downstream model to identify critical directions and reshapes isotropic LDP noise into an anisotropic distribution, preserving privacy while enhancing performance. Experiments on CIFAR-10-C show significant accuracy gains, especially for PrivUnit variants at ε=7.5.
arXiv:2607. 23649v1 Announce Type: new Abstract: Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction.
arXiv:2605. 05905v2 Announce Type: replace Abstract: Objective perturbation is a standard mechanism in differentially private empirical risk minimization.
The paper addresses the challenge of selecting external datasets for private transfer learning by modeling high‑dimensional regression with heterogeneous sources and a weighted ridge estimator. It relies solely on aggregated statistics and offers privacy guarantees under $ ho$‑zero‑concentrated differential privacy for labels or both features and labels. A deterministic equivalent of test error is derived, enabling optimization of hyperparameters and decision‑making about the utility of private external data without accessing individual records.