Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

22,941 stories · RSS feed

arXiv AI
Jun 8

MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models

arXiv:2510. 11014v2 Announce Type: replace-cross Abstract: Autonomous robots often view rooms only partially, through a doorway, where the walls and scene structure hide the geometry and task-relevant semantics needed for safe navigation and goal-directed action.

By Subhransu S. Bhattacharjee, Hao Lu, Dylan Campbell, Rahul Shome
arXiv AI
Jun 8

Linear Ordering Problem: Time for a Change

arXiv:2605. 31051v2 Announce Type: replace-cross Abstract: The Linear Ordering Problem (LOP) is a fundamental combinatorial optimization problem with important applications in areas such as economics, social choice, and machine learning.

By Fabrizio Fagiolo, Marco Baioletti, Valentino Santucci
arXiv Machine Learning
Jun 8

CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning

arXiv:2606. 07031v1 Announce Type: new Abstract: Self-supervised learning (SSL) for time-series representation learning is dominated by two paradigms: contrastive methods, which face challenges in constructing positive or negative pairs, and masking-based methods, which disrupt the temporal continuity of time-series signals.

By Jaehoon Lee, Sunghyun Sim
arXiv AI
Jun 8

FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

arXiv:2606. 06547v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written.

By Haoyu Huang, Linlin Yang, Sheng Xu, Boyu Liu, Guodong Guo, Zhongqian Fu, Hang Zhou, Baochang Zhang
arXiv Machine Learning
Jun 8

ScatterPrism: convergence for generative simulation and inverse problems in particle and nuclear physics

arXiv:2604. 01313v2 Announce Type: replace Abstract: High-fidelity simulations and complex inverse problems, such as detector modeling and unfolding, are computationally intensive bottlenecks across subatomic physics, yet essential for accurate physical interpretation.

By Zeyu Xia, Tyler Kim, Trevor Reed, Judy Fox, Geoffrey Fox, Adam Szczepaniak
arXiv AI
Jun 8

CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data

arXiv:2604. 03779v2 Announce Type: replace-cross Abstract: Diffusion models have excelled at generative tasks for both continuous and token-based domains, but their application to discrete ordinal data remains underdeveloped.

By Renzo G. Soatto, Anders Hoel, Greycen Ren, Shorna Alam, Stephen Bates, Nikolaos P. Daskalakis, Caroline Uhler, Maria Skoularidou
arXiv Machine Learning
Jun 8

Adaptive Conditional Forest Sampling for Spectral Risk Optimisation under Decision-Dependent Uncertainty

arXiv:2603. 12507v2 Announce Type: replace Abstract: Minimising a spectral risk objective, defined as a weighted combination of expected cost and Conditional Value-at-Risk (CVaR), is challenging when the uncertainty distribution is decision-dependent, making both surrogate modelling and simulation-based ranking sensitive to tail estimation error.

By Marcell T. Kurbucz
arXiv AI
Jun 8

Evaluating AI-based Scientific Knowledge Synthesis with Epidemiological Systematic Reviews

arXiv:2603. 22327v2 Announce Type: replace-cross Abstract: Systematic literature reviews (SLRs) are a demanding and high-stakes form of scientific knowledge synthesis that remains underspecified as an evaluation setting for large language models (LLMs).

By Shreyansh Padarha, Ryan Othniel Kearns, Tristan Naidoo, Lingyi Yang, {\L}ukasz Borchmann, Piotr B{\L}aszczyk, Christian Morgenstern, Ruth McCabe, Sangeeta Bhatia, Philip H. Torr, Jakob Foerster, Scott A. Hale, Thomas Rawson, Anne Cori, Elizaveta Semenova, Adam Mahdi