arXiv Machine Learning By Jeffrey G. Wang, Jason Wang, Marvin Li, Seth Neel

CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

Read the original on arXiv Machine Learning →

arXiv:2606. 17464v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 11

Black-Box Membership Inference via Word-Level Probability Estimation

The paper introduces Word-level Probability MIA (WPMIA), a black-box membership inference attack that estimates word-level generation probabilities via Monte Carlo sampling and local kernel smoothing, then aggregates them into a sequence-level likelihood estimator. By conditioning on different prefixes, WPMIA amplifies distributional differences between member and non-member texts, outperforming existing black-box baselines on open-source LLMs and achieving an average TPR@5%FPR of 42.0 on proprietary models such as GPT‑5‑Chat, Gemini‑2.5‑Flash, and Claude‑4.5‑Haiku.

By Shengjie Niu, Yeheng Ge, Jian Huang
arXiv Machine Learning
Jun 2

Causal Evaluation of Membership Inference Attacks

arXiv:2602. 02819v4 Announce Type: replace Abstract: Membership Inference Attacks (MIAs) aim to distinguish training points (members) from unseen data (non-members), and are widely used to quantify memorization and assess privacy risks.

By Mathieu Even, Cl\'ement Berenfeld, Linus Bleistein, Tudor Cebere, Julie Josse, Aur\'elien Bellet
arXiv Machine Learning
Sep 14

Membership Inference via Pairwise Likelihood Ratios

The paper introduces Pairwise Likelihood MIA (PL‑MIA), a unified membership inference attack that combines a Gaussian likelihood‑ratio statistic with population calibration and the Cauchy combination test. PL‑MIA generates p‑values from pairwise comparisons between a query point and reference points, then aggregates these continuous signals using the Cauchy test to preserve evidence strength. Experiments show that PL‑MIA surpasses strong baselines, boosting true positive rates by over 25% in low‑false‑positive settings, thereby validating the theoretical advantages of the proposed statistical framework.

By Shengjie Niu, Zebin Yun, Yeheng Ge, Jian Huang