arXiv Machine Learning

Did We Actually Fix It? An Independent Adversarial Stress-Test of Post-Point-Adjustment Evaluation Metrics for Time-Series Anomaly Detection

arXiv:2607. 11969v1 Announce Type: cross Abstract: Point-adjustment (PA), long the default scoring protocol in time-series anomaly detection (TSAD), was shown by Kim et al.

arXiv Machine Learning

Sep 22

Measuring the Checker: Mutation Analysis for GPU-Kernel Benchmark Oracles

The paper introduces mutation analysis as a metric for evaluating GPU‑kernel benchmark oracles, injecting over ten thousand faults into verified CUDA implementations of 188 KernelBench problems. It shows that the current official checkers miss 16.9% of faults, with precision faults being especially problematic, and demonstrates that optimized test suites can achieve 98% detection with only two inputs per problem. The study also reveals flaws in existing patches and a fuzzing recipe that incorrectly rejects correct kernels 107 times.

By Mingzhe Du, Anh Tuan Luu, Dong Huang, See-Kiong Ng

llms benchmarks

arXiv AI

Jun 16

Auditing Reward Hackability in Code RL Training Environments

arXiv:2606. 16062v1 Announce Type: new Abstract: We measure the rate at which code RL environments accept incorrect solutions as correct.

By Shreshth Rajan

llms reinforcement-learning

arXiv Machine Learning

Sep 22

Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance

arXiv:2605.01699v4 Announce Type: replace Abstract: Recent attacks show that behavioural unlearning of large language models leaves internal traces recoverable by adversarial probes. We characterise...

By Anamika Paul Rupa, Anietie Andy

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 10

What Demonstration Curation Metrics Do to Your Policy

arXiv:2606. 10229v1 Announce Type: cross Abstract: We study whether demonstration-curation metrics that detect defective training episodes also improve the downstream behavior-cloning policy that trains on the curated data.

By Aarav Bedi

benchmarks

arXiv AI

Jun 19

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

arXiv:2606. 20502v1 Announce Type: cross Abstract: Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved.

By Arastoo Zibaeirad, Marco Vieira

llms fine-tuning benchmarks

arXiv AI

Aug 6

Just Repair: A Minimal Denoising Network for Time Series Anomaly Detection

arXiv:2604. 17388v3 Announce Type: replace-cross Abstract: Time series anomaly detectors have grown steadily more complex, incorporating attention mechanisms, adversarial training, and stochastic latent variables.

By Kadir-Kaan \"Ozer, Ren\'e Ebeling, Markus Enzweiler

benchmarks safety

arXiv AI

Jul 23

Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

arXiv:2607. 19442v1 Announce Type: cross Abstract: Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes.

By Sen Yang, Yuen-Hei Yeung

benchmarks

arXiv Machine Learning

Jun 29

Test-Input Generation for Tensor Programs: What Actually Finds Kernel Bugs

arXiv:2606. 27396v1 Announce Type: cross Abstract: Test-input generation for tensor kernels is folkloric.

By Dipankar Sarkar

llms safety

arXiv Machine Learning

Jul 14

When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR

arXiv:2607. 11022v1 Announce Type: new Abstract: The test suites used as RLVR rewards for code have natural false positives: per-task, persistent, asymmetric errors that accept the same wrong programs every time they appear, unlike the symmetric or resampled noise assumed by existing noise-robustness analyses.

By Chuyifei Zhang

arXiv Machine Learning

Jul 9

Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection

arXiv:2607. 07146v1 Announce Type: new Abstract: The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time is the resource the effort exists to conserve.

By Joao Pinelo, Joao Goncalves, Arun Shukla, Adriana Santos-Ferreira

arXiv AI

Sep 21

LLM-Generated Feature Pools for Time Series Anomaly Detection

arXiv:2609.21801v1 Announce Type: new Abstract: We study how far a simple statistical pipeline can go on univariate time series anomaly detection under a strict selection protocol. The method extract...

By Youssef Attia El Hili, Malik Tiomoko, Corinne Ancourt

llms multimodal benchmarks

arXiv Machine Learning

Aug 11

Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify

arXiv:2608. 08008v1 Announce Type: new Abstract: Process reward models (PRMs) score intermediate reasoning steps and are widely used for search, ranking, and training, but optimization can exploit these learned proxies by increasing reward while turning correct reasoning into incorrect reasoning.

By Ibne Farabi Shihab, Fariya Afrin

fine-tuning safety

Related stories