arXiv Machine Learning June 2, 2026 By Simon Roth

Which Leakage Types Matter? A Quantitative Landscape Across 2,047 Benchmark Datasets

Read the original on arXiv Machine Learning →

arXiv:2604. 04199v2 Announce Type: replace Abstract: Twenty-eight within-subject counterfactual experiments across 2,047 iid tabular datasets, plus a boundary experiment on 129 temporal datasets, measure the severity of four data leakage classes in machine learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

benchmarks

arXiv Machine Learning

Aug 5

Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores

arXiv:2608. 02985v1 Announce Type: new Abstract: The standard check for contamination in LLM backtests is simple: compare scores before and after the training cutoff.

By Zeyu Zhang, Bradly C. Stadie

llms

arXiv Machine Learning

Jun 11

A prior-free blind detection of information leakage from model predictions

arXiv:2606. 11267v1 Announce Type: new Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise.

By Laurence A. Jacobs

arXiv Machine Learning

Aug 4

Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models

arXiv:2608. 00144v1 Announce Type: new Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines separate members from non-members from surface text alone.

By Victor Maricato

llms safety

arXiv Machine Learning

5d ago

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

arXiv:2607. 16811v4 Announce Type: replace Abstract: Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension.

By Behnam Asadi

benchmarks

arXiv Machine Learning

Jul 2

How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification

arXiv:2607. 01025v1 Announce Type: cross Abstract: Radio-frequency (RF) sensing is a central modality for counter-unmanned-aerial-system (counter-UAS) defence because it exploits the control, telemetry, and video links between a drone and its operator.

By David Shulman

robotics benchmarks

arXiv AI

Jul 14

On the Necessity of Output Distribution Reweighting for Effective Class Unlearning

arXiv:2506. 20893v5 Announce Type: replace-cross Abstract: In this paper, we reveal a significant shortcoming in class unlearning evaluations: overlooking the underlying class geometry can cause information leakage about the forgotten class.

By Ali Ebrahimpour-Boroojeny, Yian Wang, Hari Sundaram

fine-tuning benchmarks safety

Related stories

Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores

A prior-free blind detection of information leakage from model predictions

Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification

On the Necessity of Output Distribution Reweighting for Effective Class Unlearning