arXiv Machine Learning August 14, 2026 By Florian Braun

Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection

Read the original on arXiv Machine Learning →

arXiv:2608. 12652v1 Announce Type: cross Abstract: Benchmark contamination is diagnosed today with n-gram overlap, with likelihood-based membership inference, or with canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at dataset release.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

arXiv:2608. 07914v1 Announce Type: new Abstract: Behavioral contamination detectors can return "no evidence" either because a benchmark is clean or because the audit has little power.

By Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma

benchmarks

arXiv AI

Jul 23

Decodable but Not Detectable: A Leakage Fingerprint for Near-OOD Benchmarks

arXiv:2607. 19393v1 Announce Type: cross Abstract: While auditing a perturbation-based OOD detector on a document benchmark, we recorded an AUROC of 0.

By Vishnu Bindu Balachandran

rag benchmarks

arXiv AI

Jul 16

The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators

arXiv:2607. 13075v1 Announce Type: cross Abstract: Context can change whether a request is harmful without changing its topic or surface form.

By Dominik Schwarz

arXiv AI

Jul 28

Novel Claim or D\'ej\`a Vu? Rethinking "Contamination-Free'' Dynamic Evaluation for Multimodal Automated Fact-Checking

arXiv:2607. 23514v1 Announce Type: cross Abstract: Multimodal automated fact-checking (MAFC) verifies claims by retrieving and reasoning over external evidence.

By Haorui He, Xinwen Chen, Dacheng Wen, Reynold Cheng, Francis C. M. Lau, Yupeng Li

llms multimodal benchmarks

arXiv Machine Learning

5d ago

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

arXiv:2607. 16811v4 Announce Type: replace Abstract: Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension.

By Behnam Asadi

benchmarks

arXiv AI

1d ago

Calibrated Trust, Not Sharper Prediction: An Empirical Test of Uncertainty Fusion

arXiv:2608. 14617v1 Announce Type: cross Abstract: A recurring proposal in legal AI is to improve case-outcome prediction by fusing uncertainty tools (evidence graphs with belief propagation, sequential Bayesian odds updating, Dempster-Shafer combination, and conformal prediction) into one pipeline.

By Surya Saka

llms

Related stories

When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

Decodable but Not Detectable: A Leakage Fingerprint for Near-OOD Benchmarks

The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators

Novel Claim or D\'ej\`a Vu? Rethinking "Contamination-Free'' Dynamic Evaluation for Multimodal Automated Fact-Checking

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

Calibrated Trust, Not Sharper Prediction: An Empirical Test of Uncertainty Fusion