arXiv Machine Learning
Jul 23
Membership Inference Attacks for Unseen Classes
arXiv:2506. 06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is \emph{data auditing}, i.
On Reliability of Membership Inference Vulnerability Evaluation
The paper examines the reliability of membership inference attack (MIA) vulnerability evaluation. It identifies two weaknesses: finite‑sample bias from sampling shadow datasets from a fixed superset, and miscalibration when aggregating true positive rates across individuals at very low false positive rates. The authors propose simple fixes that avoid extra computational cost and suggest further improvements with additional computation.
Related stories
arXiv Machine Learning
Sep 14
Membership Inference via Pairwise Likelihood Ratios
The paper introduces Pairwise Likelihood MIA (PL‑MIA), a unified membership inference attack that combines a Gaussian likelihood‑ratio statistic with population calibration and the Cauchy combination test. PL‑MIA generates p‑values from pairwise comparisons between a query point and reference points, then aggregates these continuous signals using the Cauchy test to preserve evidence strength. Experiments show that PL‑MIA surpasses strong baselines, boosting true positive rates by over 25% in low‑false‑positive settings, thereby validating the theoretical advantages of the proposed statistical framework.
arXiv Machine Learning
Jun 17
CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
arXiv:2606. 17464v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties.
arXiv Machine Learning
Aug 17
Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA
arXiv:2603. 11799v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models.
arXiv Machine Learning
Jun 24
LoMime: Query-Efficient Membership Inference using Model Extraction in Label-Only Settings
arXiv:2602. 18934v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) threaten the privacy of machine learning models by revealing whether a specific data point was used during training.
arXiv Machine Learning
Sep 18
Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents
The paper introduces AutoMIA, a framework that uses large language model agents to automatically design and implement new membership inference attack (MIA) signal computations. By systematically exploring a wide range of attack strategies, AutoMIA discovers novel MIAs tailored to specific target models and datasets, achieving up to a 0.18 absolute improvement in AUC over existing methods. This demonstrates that LLM agents can serve as an effective and scalable approach for creating state‑of‑the‑art MIAs.
arXiv Machine Learning
Aug 11
Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators
arXiv:2603. 10937v2 Announce Type: replace Abstract: The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive domains such as healthcare, finance, and demography.
arXiv AI
Jun 19
Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software
arXiv:2606. 20502v1 Announce Type: cross Abstract: Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved.
arXiv AI
Jul 21
Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning
arXiv:2607. 16620v1 Announce Type: cross Abstract: Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems.
arXiv Machine Learning
Jul 3
From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection
arXiv:2512. 10485v2 Announce Type: replace-cross Abstract: Vulnerability detection methods based on deep learning (DL) have shown strong performance on benchmark datasets, yet their real-world effectiveness remains underexplored.
arXiv Machine Learning
Jun 9
SoK: Reconstruction Attacks on Synthetic Tabular Data (Insights from Winning the NIST CRC)
arXiv:2606. 08372v1 Announce Type: cross Abstract: Synthetic data is increasingly promoted as a privacy-preserving substitute for releasing sensitive tabular records, yet its central adversarial threat ("reconstruction", the recovery of an individual's hidden attribute values from a synthetic release and a handful of known quasi-identifiers) has been studied only in scattered, hard-to-compare settings.
arXiv AI
Jul 14
On the Necessity of Output Distribution Reweighting for Effective Class Unlearning
arXiv:2506. 20893v5 Announce Type: replace-cross Abstract: In this paper, we reveal a significant shortcoming in class unlearning evaluations: overlooking the underlying class geometry can cause information leakage about the forgotten class.