Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,285 stories · RSS feed

arXiv Machine Learning
Jul 2

Timesynth: A Temporal Fidelity Framework for Health Signal Digital Twins

arXiv:2607. 00431v1 Announce Type: new Abstract: Forecasting models for health-signal digital twins must preserve the oscillatory, frequency, phase, and state-transition dynamics of physiological signals, yet the pointwise metrics used to benchmark them cannot detect when these fundamental properties are lost.

By Md Rakibul Haque, Shireen Elhabian, Warren Woodrich Pettine
arXiv AI
Jul 2

VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement

arXiv:2607. 00446v1 Announce Type: cross Abstract: As video corpora continue to expand in both scale and task complexity, there is increasing demand for approaches that retrieve relevant videos from large-scale corpora (inter-video reasoning) and subsequently perform fine-grained, query-conditioned tasks (intra-video reasoning) within the retrieved content, such as temporal grounding.

By Seohyun Lee, Seoung Choi, Dohwan Ko, Jongha Kim, Hyunwoo J. Kim
arXiv AI
Jul 2

PaAno: Patch-Based Representation Learning for Time-Series Anomaly Detection

arXiv:2602. 01359v3 Announce Type: replace-cross Abstract: Although recent studies on time-series anomaly detection have increasingly adopted ever-larger neural network architectures such as transformers and foundation models, they incur high computational costs and memory usage, making them impractical for real-time and resource-constrained scenarios.

By Jinju Park, Seokho Kang
arXiv Machine Learning
Jul 2

Group-invariant Coresets for Data-efficient Active Learning

arXiv:2607. 01089v1 Announce Type: cross Abstract: Active learning reduces labeling cost by querying the most informative unlabeled samples, but standard coreset methods ignore known data symmetries and can waste budget on transformed versions of the same instance.

By L. C. Ayres, J. C. M. Bermudez, S. J. M. de Almeida, R. A. Borsoi
arXiv Machine Learning
Jul 2

The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology

arXiv:2607. 01033v1 Announce Type: new Abstract: Model organisms (MOs) - language models trained to exhibit undesired or unnatural behaviours - are frequently used as testbeds for evaluating white-box interpretability techniques.

By Andrzej Szablewski, Gabriel Konar-Steenberg, Raffaello Fornasiere, Nikita Menon, Stefan Heimersheim
arXiv Machine Learning
Jul 2

IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video

arXiv:2603. 16432v3 Announce Type: replace-cross Abstract: Unsupervised physical parameter estimation from video lacks a common benchmark: existing methods evaluate on non-overlapping synthetic data, the sole real-world dataset is restricted to single-body systems, and no established protocol addresses governing-equation identification.

By Rasul Khanbayov, Mohamed Rayan Barhdadi, Erchin Serpedin, Hasan Kurban