Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,452 stories · RSS feed

arXiv Machine Learning
Jun 19

Kolmogorov-Arnold Reservoir Computing

arXiv:2606. 19984v1 Announce Type: new Abstract: Reservoir computing offers a lightweight framework for forecasting dynamical systems but may struggle to capture long-range dependencies due to limited representational capacity.

By Juntian Huang, Jurgen Kurths, Ying Tang
arXiv Machine Learning
Jun 19

Utility-Aware DRL-Based TXOP Adaptation for NR-U and Wi-Fi Coexistence Networks

arXiv:2605. 00457v4 Announce Type: replace-cross Abstract: The coexistence of NR-U and Wi-Fi in the unlicensed spectrum introduces a challenging resource management problem, where heterogeneous channel access mechanisms can lead to unbalanced spectrum utilization and severe Wi-Fi performance degradation.

By Po-Heng Chou, Yi-Fang Yu, Shou-Yu Chen, Chiapin Wang
arXiv Machine Learning
Jun 19

We Need to Rethink Benchmarking in Anomaly Detection

arXiv:2507. 15584v2 Announce Type: replace Abstract: Despite the continuous proposal of new anomaly detection algorithms and extensive benchmarking efforts, progress seems to stagnate, with only minor performance differences between established baselines and new algorithms.

By Philipp R\"ochner, Simon Kl\"uttermann, Kevin Kammler, Franz Rothlauf, Emmanuel M\"uller, Daniel Schl\"or
arXiv Machine Learning
Jun 19

Evaluating deep learning models for fault diagnosis of a rotating machinery with epistemic and aleatoric uncertainty

arXiv:2412. 18980v2 Announce Type: replace Abstract: Uncertainty-aware deep learning (DL) models recently gained attention in fault diagnosis as a way to promote the reliable detection of faults when out-of-distribution (OOD) data arise from unseen faults (epistemic uncertainty) or the presence of noise (aleatoric uncertainty).

By Reza Jalayer, Masoud Jalayer, Andrea Mor, Carlotta Orsenigo, Carlo Vercellis
arXiv AI
Jun 19

Too long; didn't solve

arXiv:2604. 07593v2 Announce Type: replace Abstract: Mathematical benchmarks consisting of a range of mathematics problems are widely used to evaluate the reasoning abilities of large language models, yet little is known about how their structural properties influence model behaviour.

By Luc\'ia M. Cabrera, Isaac Saxton-Knight, Jocelyn D'Arcy
arXiv Machine Learning
Jun 19

Spectral Retrieval-Augmented Time-Series Forecasting

arXiv:2606. 19412v1 Announce Type: new Abstract: Time series forecasting leverages historical patterns to predict future values, but traditional methods face challenges when dealing with complex, non-stationary patterns that are difficult to memorize during training.

By Huu Hiep Nguyen, Minh Hoang Nguyen, Dung Nguyen, Hung Le