The paper addresses a flaw in differential privacy audits where selection bias in Membership Inference Attacks leads to overestimated lower bounds on privacy leakage. By applying statistical correction methods, the authors demonstrate that uncorrected audits can violate theoretical privacy guarantees up to four times more often than claimed, while corrected audits maintain valid bounds at the nominal confidence level.
By Valentin Dorseuil (DI-ENS), Meenatchi Sundaram Muthu Selva Annamalai (UCL), Jamal Atif (CMAP), Olivier Capp\'e (DI-ENS), Nicolas Papernot (Inria)
arXiv:2605. 07674v2 Announce Type: replace-cross Abstract: Regulatory audits of AI systems increasingly rely on differential privacy (DP) to protect training data and model internals.
By Florian A. D. Burnat
arXiv:2608.28934v1 Announce Type: new
Abstract: Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In...
By Saloni Modi, Srivi Balaji, Yusong Zhu, Gautam Kamath, Kevin Tian
arXiv:2606. 12733v2 Announce Type: replace Abstract: Privacy auditing provides an important safeguard by estimating the actual information leaked by a model, thus ensuring that theoretical privacy guarantees hold in practice.
By Adya Agrawal, Yu Wei, Jaspal Singh, Malik Magdon-Ismail, Vassilis Zikas
arXiv:2606. 14518v1 Announce Type: new Abstract: The removal of learned data from Machine Learning models through Machine Unlearning (MU) has been widely studied; however, there has yet to be an agreed-upon scheme for auditing MU.
By Liou Tang, James Joshi, Ashish Kundu
The paper introduces DISCERN, a two-tier protocol for certifying that updates to production models do not increase risk. It first uses unlabeled data to detect benign updates based on disagreement rates, then selectively labels only disagreements through an anytime-valid confidence sequence. The method achieves finite-sample validity with label-complexity bounds of order ρ²/ε², demonstrating significant label savings and strong empirical performance across 14,000+ audit streams.
By Vishnu Bindu Balachandran