arXiv:2608. 12652v1 Announce Type: cross Abstract: Benchmark contamination is diagnosed today with n-gram overlap, with likelihood-based membership inference, or with canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at dataset release.
By Florian Braun
arXiv:2607. 16811v4 Announce Type: replace Abstract: Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension.
By Behnam Asadi
arXiv:2606. 13780v1 Announce Type: cross Abstract: Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.
By Jack Y. Araz, Michael Spannowsky
The study evaluates a convolutional neural network (CNN) technosignature detector trained on Breakthrough Listen data by injecting 15,000 synthetic signals of varied morphologies and amplitudes into real Green Bank Observatory backgrounds. Results show that the detector’s recall decreases with increasing signal brightness and fails to detect certain signal morphologies, while some interference patterns produce false positive rates comparable to its baseline recall. The analysis reveals that the detector only processes ON panels, lacking the OFF panel information needed to distinguish genuine extraterrestrial signals from terrestrial interference, which explains both the false alarms and missed detections.
By Omer Imran
arXiv:2608. 07914v1 Announce Type: new Abstract: Behavioral contamination detectors can return "no evidence" either because a benchmark is clean or because the audit has little power.
By Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma
The paper demonstrates that the way evaluation streams are assembled in streaming intrusion‑detection benchmarks—by interleaving, pooling, or replaying network captures—acts as an uncontrolled experimental variable that can significantly alter performance metrics. In the CICIDS2017 benchmark, reordering the same set of records under a fixed split changes the held‑out samples’ overlap, prevalence, and even reverses the ranking of two deterministic scorers. Similar effects are observed in the LITNET‑2020 benchmark, where pooling disjoint captures yields a single operating point that masks large variations in per‑capture prevalences, and minor changes in batch composition can shift reported AUC‑PR values by a few thousandths.
By Michel A. Youssef