Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,090 stories · RSS feed

arXiv Machine Learning

Jun 5

EML-CD: Causal Mechanism Recovery via EML Symbolic Trees in Structure Learning

arXiv:2606. 05942v1 Announce Type: cross Abstract: Neural network (NN)-based nonlinear causal discovery methods recover DAG structure but leave each causal mechanism as a black box.

By Sota Asanuma

benchmarks

arXiv Machine Learning

Jun 5

HoT-SSM:Higher-order Temporal Knowledge Graph Reasoning with State Space Models for Health Care

arXiv:2606. 05994v1 Announce Type: new Abstract: Medical knowledge graphs (MKGs) infused with clinical knowledge have been increasingly used to model electronic health records (EHRs) to support interpretable predictions in healthcare domain.

By Thummaluru Siddartha Reddy, Vempalli Naga Sai Saketh, Yash Punjabi, Mahesh Chandran

benchmarks

arXiv Machine Learning

Jun 5

Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder

arXiv:2606. 05981v1 Announce Type: cross Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1-step distilled student, the text encoder becomes the critical path.

By Yoshiyuki Ootani

llms diffusion efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data

arXiv:2606. 06328v1 Announce Type: new Abstract: In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because electrodes detach or an entire respiratory channel is unavailable during overnight monitoring.

By Ziwen Kan, Wugeng Zheng, Tianlong Chen, Song Wang

rag diffusion multimodal benchmarks

arXiv Machine Learning

Jun 5

Toto 2.0: Time Series Forecasting Enters the Scaling Era

arXiv:2605. 20119v2 Announce Type: replace Abstract: We show that time series foundation models scale: a single training recipe produces reliable forecast-quality improvements from 4M to 2.

By Emaad Khwaja, Chris Lettieri, Gerald Woo, Eden Belouadah, Marc Cenac, Guillaume Jarry, Enguerrand Paquin, Xunyi Zhao, Viktoriya Zhukov, Othmane Abou-Amal, Chenghao Liu, Ameet Talwalkar, David Asker

benchmarks

arXiv Machine Learning

Jun 5

$p$-adic Bi-Filtrations for Topological Machine Learning on Genomic Sequences

arXiv:2606. 06117v1 Announce Type: cross Abstract: We introduce pVR, a topological machine learning framework for alignment-free genomic sequence classification that combines $p$-adic numbers with topological data analysis.

By Tirtharaj Dash, Gunja Sachdeva

llms rag benchmarks safety

arXiv Machine Learning

Jun 5

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs

arXiv:2602. 09574v2 Announce Type: replace-cross Abstract: Tree-search decoding is an effective form of test-time scaling for large language models (LLMs), but real-world deployment often imposes a fixed per-query token budget that varies across settings.

By Sora Miyamoto, Daisuke Oba, Naoaki Okazaki

llms benchmarks

arXiv Machine Learning

Jun 5

Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and global attention models

arXiv:2508. 19006v2 Announce Type: replace-cross Abstract: This study investigates the pre-trained RNN attention models with the mainstream attention mechanisms, such as additive attention, Luong's three attentions, global self-attention and sliding window sparse attention, for the empirical asset pricing research on the top 420 large-cap US stocks.

By Shanyan Lai

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Reward Learning through Ranking Mean Squared Error

arXiv:2601. 09236v3 Announce Type: replace Abstract: Reward design remains a significant bottleneck in applying reinforcement learning (RL) to real-world problems.

By Chaitanya Kharyal, Calarina Muslimani, Matthew E. Taylor

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 5

Symmetric Divergence and Normalized Similarity: A Unified Topological Framework for Representation Analysis

arXiv:2606. 06342v1 Announce Type: cross Abstract: Topological Data Analysis (TDA) offers a principled, intrinsic lens for comparing neural representations.

By Yan Wang, Tianyang Hu

llms benchmarks

arXiv Machine Learning

Jun 5

When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories

arXiv:2606. 05414v1 Announce Type: cross Abstract: Early failure alerting requires deciding, while a dialog or agent trajectory is still unfolding, whether to flag it as likely to fail.

By Avinash Baidya, Xinran Liang, Ruocheng Guo, Xiang Gao, Kamalika Das

llms agents benchmarks

arXiv Machine Learning

Jun 5

Performance Evaluation of GraphCast for Medium-Range Weather Forecasting over Brazil

arXiv:2606. 06348v1 Announce Type: new Abstract: The paradigm of global weather forecasting is rapidly shifting with the emergence of Machine Learning Weather Prediction models (MLWP).

By Wolfgang R. Rowell Jr., Lucas S. Kupssinsk\"u

benchmarks

arXiv Machine Learning

Jun 5

SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions

arXiv:2604. 08477v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved reasoning in formal domains such as mathematics and code, but extending these gains beyond STEM remains challenging.

By Ashima Suvarna, Kendrick Phan, Mehrab Beikzadeh, Hritik Bansal, Saadia Gabriel

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

A Differentiable Framework for Full and Phaseless Data Inversion Using Neural Implicit Contrast-Source Representation

arXiv:2508. 10555v2 Announce Type: replace-cross Abstract: In this study, we extend the contrast source inversion to a fully differentiable, unsupervised framework based on a neural implicit representation of the contrast source.

By Haoran Sun, Daoqi Liu, Hongyu Zhou, Maokun Li, Shenheng Xu, Fan Yang

benchmarks

arXiv Machine Learning

Jun 5

GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

arXiv:2606. 05441v1 Announce Type: new Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones.

By Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali, Gianfranco Doretto, Donald A. Adjeroh

nlp benchmarks

arXiv Machine Learning

Jun 5

3D Underwater Path Planning via Generative Flow Field Surrogates

arXiv:2606. 06077v1 Announce Type: cross Abstract: Autonomous underwater vehicle (AUV) launch and recovery (LAR) into the hull of an advancing host platform requires traversal of a complex, three-dimensional propeller wake whose hydrodynamic structure cannot be characterised by a uniform current model.

By Zachary Cooper-Baldock, Paulo E. Santos, Russell S. A. Brinkworth, Karl Sammut

agents robotics benchmarks safety

arXiv Machine Learning

Jun 5

SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter

arXiv:2606. 05800v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) often adopts GRPO-style group-relative updates, sampling multiple rollouts per prompt to construct normalized learning signals.

By Powei Chang, Jinpeng Zhang, Chaoqun Sun, MiniWell Tsao, Lianrui Li, Jianxiang Xiang, Chenyu Wang, Yukang Gao, Dongying Kong

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Towards Label-Noise Resistant Learning via Optimal Brain Damage Masking

arXiv:2508. 09697v3 Announce Type: replace Abstract: Noisy labels are inevitable in real-world scenarios.

By Xinlei Zhang, Fan Liu, Chuanyi Zhang, Fan Cheng, Qian Li, Yuhui Zheng

efficiency benchmarks

arXiv Machine Learning

Jun 5

Scalable Reinforcement Learning via Adaptive Batch Scaling

arXiv:2605. 21557v2 Announce Type: replace-cross Abstract: Conventional wisdom holds that large-batch training is fundamentally incompatible with Reinforcement Learning (RL) - beyond a modest threshold, increasing batch sizes typically yields diminishing returns or performance degradation due to the inherent non-stationarity of the data distribution.

By Jongchan Park

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.

By Yongzhong Xu

llms benchmarks

Get the daily briefing