Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning

Oct 7

Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation

arXiv:2511.11500v3 Announce Type: replace Abstract: Modern language models fail a fundamental requirement of trustworthy intelligence: knowing when not to answer. Despite achieving impressive accurac...

By Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan Li

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 7

Stochastic Siamese MAE Pretraining for Longitudinal Medical Images

arXiv:2512.23441v2 Announce Type: replace Abstract: Temporally aware image representations are crucial for capturing disease progression in 3D volumes of longitudinal medical datasets. However, recen...

By Taha Emre, Arunava Chakravarty, Thomas Pinetz, Dmitrii Lachinov, Martin J. Menten, Hendrik Scholl, Sobha Sivaprasad, Daniel Rueckert, Andrew Lotery, Stefan Sacu, Ursula Schmidt-Erfurth, Hrvoje Bogunovi\'c

benchmarks

arXiv Machine Learning

Oct 7

From Dual Tracking to Clipping: Provably Faster Distributionally Robust Multi-Objective Optimization

arXiv:2605.05660v3 Announce Type: replace Abstract: Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, most exis...

By Yufeng Yang, Fangning Zhuo, Ziyi Chen, Heng Huang, Yi Zhou

benchmarks safety

arXiv Machine Learning

Oct 7

Particle Monte Carlo Tree Search

arXiv:2605.08982v4 Announce Type: replace Abstract: Monte Carlo Tree Search (MCTS) is a widely used approach for policy improvement and action selection in Reinforcement Learning. Due to its sequenti...

By Yaniv Oren, Viliam Vadocz, Joery A. de Vries, Wendelin B\"ohmer, Matthijs T. J. Spaan, Hendrik Baier

llms reinforcement-learning benchmarks

arXiv Machine Learning

Oct 7

A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Models

arXiv:2608.20980v2 Announce Type: replace Abstract: Graph neural networks (GNNs) are routinely employed for spatiotemporal forecasting, yet their performance across widely used benchmark datasets is...

By Kenneth Martin, Simon Heilig, Asja Fischer, Michel F. C. Haddad, Adam M. Sykulski, Moshe Eliasof

benchmarks

arXiv Machine Learning

Oct 7

PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction

arXiv:2609.34054v2 Announce Type: replace Abstract: Multi-LoRA agent systems enable efficient role specialization by sharing a common backbone model. However, each agent repeatedly processes the grow...

By Hyesung Jeon, Hyeongju Ha, Jae-Joon Kim

agents fine-tuning efficiency benchmarks

arXiv Machine Learning

Oct 7

Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfer

arXiv:2610.04946v2 Announce Type: replace Abstract: Cross-system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, a...

By Qingyang Zhang

llms rag benchmarks

arXiv Machine Learning

Oct 7

Lossy Compression of PDE Training Inputs: Field Reconstruction Error Does Not Order the Cost to a Trained Operator

arXiv:2610.06095v2 Announce Type: replace Abstract: Operator-learning benchmarks are stored at full precision and have grown to terabyte scale. Rate-distortion theory says how many bits the stored fi...

By Huy Hoang Le

benchmarks

arXiv Machine Learning

Oct 7

Generalizable Dense Reward for Long-Horizon Robotic Tasks

arXiv:2604.00055v2 Announce Type: replace-cross Abstract: Existing robotic foundation policies are trained primarily via large-scale imitation learning. While such models demonstrate strong capabilit...

By Silong Yong, Stephen Sheng, Carl Qi, Xiaojie Wang, Evan Sheehan, Anurag Shivaprasad, Yaqi Xie, Katia Sycara, Yesh Dattatreya

llms reinforcement-learning robotics fine-tuning multimodal benchmarks