arXiv Machine Learning June 16, 2026 By Jingwen Zhou, Mingzhe Wang

False Sense of Safety in Selective Signal Classification: Auditing Bound Tightness and Exchangeability for Risk Control

Read the original on arXiv Machine Learning →

arXiv:2606. 15153v1 Announce Type: new Abstract: Selective prediction with distribution-free risk control promises that, with confidence 1-delta over the calibration draw, the error rate of accepted inputs stays below a user budget alpha.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

safety

arXiv AI

Aug 11

When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

arXiv:2608. 07914v1 Announce Type: new Abstract: Behavioral contamination detectors can return "no evidence" either because a benchmark is clean or because the audit has little power.

By Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma

benchmarks

arXiv Machine Learning

Jun 11

A prior-free blind detection of information leakage from model predictions

arXiv:2606. 11267v1 Announce Type: new Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise.

By Laurence A. Jacobs

arXiv AI

Aug 11

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

arXiv:2608. 07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target.

By Sanjeda Akter, Ibne Farabi Shihab, Anuj Sharma

rag nlp safety

arXiv AI

1d ago

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

arXiv:2608. 16147v1 Announce Type: new Abstract: Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of the method.

By Diyorbek Musaev

benchmarks

Hugging Face Trending Papers

Aug 8

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

Selective-risk certificates promise that accepted outputs meet a declared error target. We develop Fed-SRC, a score-agnostic certificate for federated, differentially private, adaptively monitored retrieval-augmented generation.

rag nlp safety

arXiv Machine Learning

Jun 15

Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches

arXiv:2606. 13780v1 Announce Type: cross Abstract: Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.

By Jack Y. Araz, Michael Spannowsky

Related stories

When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

A prior-free blind detection of information leakage from model predictions

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches