The paper shows that when machine‑learning datasets are built by applying a detector to a candidate pool, the precision of the resulting labels is largely determined by the prevalence of true positives in that pool, not just the detector’s accuracy. In a study using one detector across three different pools, phantom (false) rates varied from 81.7% to 0%, and the Bayes‑based precision estimate matched all three pools within 3.3%, whereas naive transfer of precision from high‑rate pools over‑estimated precision by 422%. The authors further demonstrate that contamination behaves as a second signal with the detector’s shape, affecting different estimators in opposite ways, and that common normalizations can produce misleading results.
arXiv:2608. 12652v1 Announce Type: cross Abstract: Benchmark contamination is diagnosed today with n-gram overlap, with likelihood-based membership inference, or with canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at dataset release.
arXiv:2607. 16811v4 Announce Type: replace Abstract: Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension.
arXiv:2606. 13780v1 Announce Type: cross Abstract: Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.
The study evaluates a convolutional neural network (CNN) technosignature detector trained on Breakthrough Listen data by injecting 15,000 synthetic signals of varied morphologies and amplitudes into real Green Bank Observatory backgrounds. Results show that the detector’s recall decreases with increasing signal brightness and fails to detect certain signal morphologies, while some interference patterns produce false positive rates comparable to its baseline recall. The analysis reveals that the detector only processes ON panels, lacking the OFF panel information needed to distinguish genuine extraterrestrial signals from terrestrial interference, which explains both the false alarms and missed detections.
arXiv:2608. 07914v1 Announce Type: new Abstract: Behavioral contamination detectors can return "no evidence" either because a benchmark is clean or because the audit has little power.
The paper demonstrates that the way evaluation streams are assembled in streaming intrusion‑detection benchmarks—by interleaving, pooling, or replaying network captures—acts as an uncontrolled experimental variable that can significantly alter performance metrics. In the CICIDS2017 benchmark, reordering the same set of records under a fixed split changes the held‑out samples’ overlap, prevalence, and even reverses the ranking of two deterministic scorers. Similar effects are observed in the LITNET‑2020 benchmark, where pooling disjoint captures yields a single operating point that masks large variations in per‑capture prevalences, and minor changes in batch composition can shift reported AUC‑PR values by a few thousandths.
The paper introduces a new taxonomy for benchmark contamination that categorizes leakage by the mitigation it defeats—direct, derivative, temporal, distributional, and acquired—covering both training‑time and evaluation‑time scenarios. It proposes a four‑field disclosure protocol to record contamination status alongside benchmark scores, and provides a JSON schema, validator, and examples. An empirical study of 41 documents using a pre‑registered instrument shows limited reporting of contamination types and variable reliability, highlighting gaps in current disclosure practices.
arXiv:2610.18904v1 Announce Type: cross Abstract: A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the val...
arXiv:2608. 15090v1 Announce Type: cross Abstract: Studies of industrial visual inspection commonly report the area under the receiver operating characteristic curve (AUROC) and the overlap between anomaly maps and defect masks.
arXiv:2607. 13075v1 Announce Type: cross Abstract: Context can change whether a request is harmful without changing its topic or surface form.
The paper presents a calibrated evidence inference model fine‑tuned from Qwen3‑4B‑Base using low‑rank adapters and a classification head to classify trial outcomes as significant increase, significant decrease, or no significant difference. Trained on 1,431 open‑access PubMed Central articles, the released model (Apache 2.0) achieves an expected calibration error of 0.017, a log loss lower than Gemma 4 31B‑it in zero‑shot settings, and a macro‑F1 of 0.925 on the test split. Accepting its top‑80 % most confident predictions yields only a 2.6 % error rate, and ablation studies show that article context influences answers while the title alone still provides useful signal.
arXiv:2607. 07146v1 Announce Type: new Abstract: The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time is the resource the effort exists to conserve.