arXiv Machine Learning August 4, 2026 By Victor Maricato

Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models

Read the original on arXiv Machine Learning →

arXiv:2608. 00144v1 Announce Type: new Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines separate members from non-members from surface text alone.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 12

Leak It: Per-Document Extraction Beyond Aggregate Membership Inference

arXiv:2608. 00144v2 Announce Type: replace Abstract: Membership inference (MIA) on language models is usually summarised by aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines can separate members from non-members using surface text alone.

By Victor Maricato

llms safety

arXiv AI

Jul 7

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.

By Mohamed Aly Bouke

rag nlp benchmarks safety

arXiv Machine Learning

Jul 7

The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.

By Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann, Matthias A{\ss}enmacher

llms safety

arXiv AI

Jun 9

MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models

arXiv:2606. 07996v1 Announce Type: cross Abstract: Pretraining is fundamental to the development of Large Language Models (LLMs), yet the opacity of pretraining data complicates model analysis and raises ethical, legal, and fairness concerns.

By Kaixin Lan, Mu You, Tao Fang, Binkai Ou, Lidia S. Chao, Derek F. Wong

llms benchmarks safety

arXiv AI

Jul 2

Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

arXiv:2607. 01208v1 Announce Type: cross Abstract: Language models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale.

By Shayan Talaei, Abhinav Chinta, Devvrit Khatri, Amin Karbasi, Azalia Mirhoseini, Amin Saberi

llms efficiency safety

Hugging Face Trending Papers

Jun 22

Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models

Large Language Models (LLMs) raise growing concerns about privacy leakage and copyright compliance. Membership inference is a key tool for assessing such risks, but existing studies mainly focus on whether specific samples or sample-based data units are used for training.

llms benchmarks safety

Related stories

Leak It: Per-Document Extraction Beyond Aggregate Membership Inference

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models

Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models