CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
arXiv:2606. 17464v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties.
The paper introduces Pairwise Likelihood MIA (PL‑MIA), a unified membership inference attack that combines a Gaussian likelihood‑ratio statistic with population calibration and the Cauchy combination test. PL‑MIA generates p‑values from pairwise comparisons between a query point and reference points, then aggregates these continuous signals using the Cauchy test to preserve evidence strength. Experiments show that PL‑MIA surpasses strong baselines, boosting true positive rates by over 25% in low‑false‑positive settings, thereby validating the theoretical advantages of the proposed statistical framework.
arXiv:2606. 17464v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties.
arXiv:2509. 25003v3 Announce Type: replace Abstract: Membership inference attacks (MIAs) against Diffusion Models (DMs) raise pressing privacy concerns by revealing whether a sample was part of the training set.
arXiv:2603. 11799v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models.
arXiv:2602. 02819v4 Announce Type: replace Abstract: Membership Inference Attacks (MIAs) aim to distinguish training points (members) from unseen data (non-members), and are widely used to quantify memorization and assess privacy risks.
arXiv:2608.28934v1 Announce Type: new Abstract: Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In...
The paper introduces a shift‑aware score transport method for privacy auditing that does not require retraining the target model. It estimates the in‑distribution non‑member score distribution from out‑of‑distribution data, transports this relationship to the target covariate distribution, and applies a conservative correction to account for potential extrapolation errors. Experiments on multiple datasets with distribution shifts show that the method provides more stable privacy estimates while maintaining tighter lower bounds than existing baselines.
The paper examines the reliability of membership inference attack (MIA) vulnerability evaluation. It identifies two weaknesses: finite‑sample bias from sampling shadow datasets from a fixed superset, and miscalibration when aggregating true positive rates across individuals at very low false positive rates. The authors propose simple fixes that avoid extra computational cost and suggest further improvements with additional computation.
arXiv:2506. 06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is \emph{data auditing}, i.
The paper introduces a method to estimate a model’s vulnerability to the Likelihood Ratio Attack (LiRA) without training reference models, using only the target model’s train and test loss distributions. It shows that LiRA’s per‑sample signal can be decomposed into a variance‑ratio term and a residual mean‑shift term, and that different loss‑distribution shapes dictate which reference‑free proxy to use. Two proxies are presented: the LOSS attack TNR for heavy‑tailed losses and the LOSS attack AUC for symmetric losses, both achieving low RMSE in predicting LiRA TPR across multiple architectures and datasets.
arXiv:2603. 10937v2 Announce Type: replace Abstract: The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive domains such as healthcare, finance, and demography.
The paper introduces PSCMIA, a power side‑channel membership inference attack that can determine whether a data sample was used to train an embedded machine learning model by analyzing power traces, without needing prediction probabilities or labels. The authors evaluate PSCMIA on MNIST, FMNIST, CIFAR10, and CINIC10 datasets using fully connected and convolutional neural networks on STM32F3 and XMEGA platforms, achieving ROC‑AUC values up to 0.907 for fully connected models and close performance to probability‑vector attacks for CNNs. PSCMIA outperforms label‑only attacks in 11 of 16 configurations, showing that physical execution can leak membership information even when conventional outputs are suppressed.
arXiv:2602. 18934v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) threaten the privacy of machine learning models by revealing whether a specific data point was used during training.