Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,090 stories · RSS feed

arXiv Machine Learning

Jun 5

Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents

arXiv:2606. 05296v1 Announce Type: new Abstract: LLM agents operate in two distinct regimes: open-weight agents amenable to reinforcement learning (RL) and black-box agents whose behaviour must be controlled purely at test time.

By Dae Yon Hwang, Raunaq Suri, Valentin Villecroze, Anthony L. Caterini, Jesse C. Cresswell, No\"el Vouitsis, Brendan Leigh Ross

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Step-adaptive multimodal fusion network with multi-scale cloud feature learning for ultra-short-term solar irradiance forecasting

arXiv:2606. 06102v1 Announce Type: cross Abstract: Ultra-short-term solar irradiance prediction is critical for photovoltaic system dispatch and power grid stability.

By Jingxin Zhang Xiaoqin Wang

multimodal benchmarks

arXiv Machine Learning

Jun 5

EML-CD: Causal Mechanism Recovery via EML Symbolic Trees in Structure Learning

arXiv:2606. 05942v1 Announce Type: cross Abstract: Neural network (NN)-based nonlinear causal discovery methods recover DAG structure but leave each causal mechanism as a black box.

By Sota Asanuma

benchmarks

arXiv Machine Learning

Jun 5

Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder

arXiv:2606. 05981v1 Announce Type: cross Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1-step distilled student, the text encoder becomes the critical path.

By Yoshiyuki Ootani

llms diffusion efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

$p$-adic Bi-Filtrations for Topological Machine Learning on Genomic Sequences

arXiv:2606. 06117v1 Announce Type: cross Abstract: We introduce pVR, a topological machine learning framework for alignment-free genomic sequence classification that combines $p$-adic numbers with topological data analysis.

By Tirtharaj Dash, Gunja Sachdeva

llms rag benchmarks safety

arXiv Machine Learning

Jun 5

Learned Response-Field Inertia Operator for HEC-RAS 2D Water-Surface Elevation Prediction

arXiv:2606. 06385v1 Announce Type: new Abstract: This article presents a cross-dataset evaluation of learned native-cell surrogate models for solver-consistent water-surface elevation (WSE) prediction in HEC-RAS 2D.

By Edward Holmberg, Elias Ioup, Md Meftahul Ferdaus, Mahdi Abdelguerfi, Julian Simeonov

computer-vision benchmarks

arXiv Machine Learning

Jun 5

Reframing preprocessing selection as model-internal calibration in near-infrared spectroscopy: A large-scale benchmark of operator-adaptive PLS and Ridge models

arXiv:2605. 13587v3 Announce Type: replace-cross Abstract: Preprocessing screening is often the most expensive part of a near-infrared spectroscopy calibration workflow.

By Gregory Beurier (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Robin Reiter (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Camille No\^us (Laboratoire Cogitamus), Lauriane Rouan (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Denis Cornet (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France)

benchmarks

arXiv Machine Learning

Jun 5

Flash-WAM: Modality-Aware Distillation for World Action Models

arXiv:2606. 05254v1 Announce Type: new Abstract: World-action models (WAMs) jointly generate future video and robot actions through iterative diffusion, achieving strong performance on manipulation benchmarks but requiring tens of denoising steps, a cost that precludes real-time control.

By Arman Akbari, Ci Zhang, Arash Akbari, Lin Zhao, Yixiao Chen, Weiwei Chen, Xuan Zhang, Geng Yuan, Yanzhi Wang

diffusion robotics efficiency benchmarks

arXiv Machine Learning

Jun 5

Symmetric Divergence and Normalized Similarity: A Unified Topological Framework for Representation Analysis

arXiv:2606. 06342v1 Announce Type: cross Abstract: Topological Data Analysis (TDA) offers a principled, intrinsic lens for comparing neural representations.

By Yan Wang, Tianyang Hu

llms benchmarks

arXiv Machine Learning

Jun 5

Vision Hopfield Memory Networks

arXiv:2603. 25157v2 Announce Type: replace Abstract: Recent vision and multimodal foundation backbones, such as Transformer families and state-space models like Mamba, have achieved remarkable progress, enabling unified modeling across images, text, and beyond.

By Jianfeng Wang, Amine M'Charrak, Luk Koska, Xiangtao Wang, Daniel Petriceanu, Ruizhi Wang, Michael Bumbar, Luca Pinchetti, Thomas Lukasiewicz

llms computer-vision multimodal benchmarks safety

arXiv Machine Learning

Jun 5

The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models

arXiv:2606. 05169v1 Announce Type: new Abstract: We give a stereological theory of LLM benchmark coverage.

By Jason Z Wang

llms benchmarks

arXiv Machine Learning

Jun 5

Multimarginal flow matching with optimal transport potentials

arXiv:2606. 05327v1 Announce Type: new Abstract: Flow matching (FM) has emerged as a powerful framework for learning dynamic transport maps between two empirical distributions.

By Raghav Kansal, David Crair, Nghia Nguyen, Scott Pope, Bradley Parry

diffusion benchmarks

arXiv Machine Learning

Jun 5

MDP-GRPO: Stabilized Group Relative Policy Optimization for Multi-Constraint Instruction Following

arXiv:2606. 06058v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards is ideal for multi-constraint instruction following, yet standard group-relative policy optimization (GRPO) becomes unstable under discrete, low-dispersion rewards, where within-group reward distributions are frequently homogeneous.

By Mohammad Mahdi Salmani-Zarchi, Zahra Rahimi, Heshaam Faili, Mohammad Javad Dousti

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Central Description Length (CDL) Clustering Validation Index

arXiv:2606. 05230v1 Announce Type: cross Abstract: Selecting a clustering algorithm and its hyperparameters without labels is a common difficulty in engineering machine learning pipelines that work with unsupervised analysis of sensor, image, or process data.

By Mahdi Shamsi, Soosan Beheshti

rag benchmarks

arXiv Machine Learning

Jun 5

CausalPOI: Spatio-Temporal Graph-Based Causal Modeling for Cold-Start POI Check-in Forecasting

arXiv:2606. 05413v1 Announce Type: new Abstract: As urban environments continue to evolve rapidly, accurately modeling the dynamic behaviour of Points of Interest is essential for supporting data-driven urban planning and commercial decision-making.

By Zhaoqi Zhang, Miao Xie, Yi Li, Linyou Cai, Siqiang Luo, Gao Cong

benchmarks

arXiv Machine Learning

Jun 5

Differentiable Efficient Operator Search

arXiv:2606. 05232v1 Announce Type: new Abstract: Efficient multimodal foundation models often rely on manually designed token-reduction operators, such as pruning, merging, pooling, and adaptive reweighting.

By Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo, Weiguo Feng, Tao Huang, Cho-Jui Hsieh, Chang Xu

efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models

arXiv:2506. 11042v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a resource-efficient strategy for adapting Pretrained Foundation Models (PFMs) by learning a small number of task-specific updates $\Delta W$.

By Guangning Xu, Baoquan Zhang, Michael. K. Ng

llms nlp fine-tuning benchmarks

arXiv Machine Learning

Jun 5

LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")

arXiv:2606. 05497v1 Announce Type: new Abstract: Given the inherently multimodal nature of human experience, vision-language models (VLMs) hold substantial promise for modeling human cognition as it grows and develops with experience.

By Alvin Wei Ming Tan, David Cardinal, Tania Lorido-Botran, Laura Bravo-Sanchez, Sunny Yu, Michael C. Frank

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

arXiv:2606. 06467v1 Announce Type: cross Abstract: Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of thought.

By Yutao Sun, Yanqi Zhang, Li Dong, Jianyong Wang, Furu Wei

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Semi-Offline Reinforcement Learning for Optimized Text Generation

arXiv:2306. 09712v2 Announce Type: replace Abstract: In reinforcement learning (RL), there are two major settings for interacting with the environment: online and offline.

By Changyu Chen, Xiting Wang, Yiqiao Jin, Victor Ye Dong, Li Dong, Jie Cao, Yi Liu, Rui Yan

reinforcement-learning benchmarks

Get the daily briefing