The paper presents a new analysis of Oja's algorithm for streaming principal component analysis (PCA) that works without any eigengap assumptions, achieving near‑optimal rates and matching lower bounds. It extends the results to a Rayleigh quotient notion of approximate PCA, resolving an open question, and applies the findings to provide gap‑free differentially private PCA guarantees for sub‑Gaussian data. The analysis relies solely on a second‑moment bound of stochastic updates, avoiding the almost‑sure bounds used in previous work.
By Anming Gu, Syamantak Kumar, Kevin Tian, Chutong Yang
arXiv:2606. 00342v1 Announce Type: new Abstract: We study the problem of differentially private (DP) $k$-means clustering in Euclidean space.
By Thomas Humphries, Zinan Lin, Sergey Yekhanin
arXiv:2505. 14251v2 Announce Type: replace Abstract: We study the problem of differentially private second moment estimation and present a new algorithm that achieve strong privacy-utility trade-offs even for worst-case inputs under subsamplability assumptions on the data.
By Bar Mahpud, Or Sheffet
arXiv:2606. 17995v1 Announce Type: cross Abstract: We study the privacy of releasing posterior sample paths from a Gaussian process (GP) when the entire training set including covariates and responses is private.
By Tomasz Maciazek
arXiv:2606. 09582v1 Announce Type: new Abstract: Recent work argues for using Gaussian differential privacy (GDP) to report the privacy guarantees in privacy-preserving machine learning.
By Bogdan Kulynych, Antti Honkela
arXiv:2602. 03682v2 Announce Type: replace-cross Abstract: We analyze the Accelerated Noisy Power Method, an algorithm for Principal Component Analysis in the setting where only inexact matrix-vector products are available, which can arise for instance in decentralized PCA.
By Pierre Agui\'e, Mathieu Even, Laurent Massouli\'e
arXiv:2508.04800v2 Announce Type: replace-cross
Abstract: We introduce a novel privatization framework for high-dimensional controlled variable selection. Our framework enables rigorous False Discove...
By Yuxuan Tao, Adel Javanmard
arXiv:2608. 13229v1 Announce Type: cross Abstract: We present the mathematical foundations of linear independent component analysis (ICA) models based on standard literature in a self-contained note.
By Patrick Forr\'e
arXiv:2508. 18037v2 Announce Type: replace Abstract: Leveraging information from public data has become increasingly crucial in enhancing the utility of differentially private (DP) methods.
By Zilong Cao (The School of Mathematics, Northwest University), Hai Zhang (The School of Mathematics, Northwest University)
arXiv:2606. 01908v1 Announce Type: new Abstract: Test-time adaptation (TTA) can reduce error on new and different data by updating the model on these inputs during inference.
By Zefeng Li, Qiaoyue Tang, Mathias Lecuyer, Evan Shelhamer
arXiv:2605. 05905v2 Announce Type: replace Abstract: Objective perturbation is a standard mechanism in differentially private empirical risk minimization.
By Daniel Cortild, Coralia Cartis
The paper introduces DP-Muon, a differentially private optimization method that incorporates matrix‑orthogonalized momentum. It employs standard global per‑example clipping and releases a single Gaussian‑noised gradient per step, with matrix and auxiliary updates treated as post‑processing. The authors analyze the mean distortion introduced when fresh Gaussian noise passes through a nonlinear matrix map, deriving exact Gaussian heat identities and showing that for a smooth Newton‑Schulz map, the conditional output bias is reduced from second to fourth order in the noise scale. They also establish matrix‑block stationarity bounds, quantify orthogonalization error, and provide criteria for improving the upper bound, while a separate inequality captures the impact of auxiliary Adam updates. Experiments on GPT‑2 at various privacy targets demonstrate that DP‑Muon configurations outperform Adam baselines in test negative log‑likelihood.
By Jihwan Kim, Chenglin Fan