Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 5

Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder

arXiv:2606. 05981v1 Announce Type: cross Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1-step distilled student, the text encoder becomes the critical path.

By Yoshiyuki Ootani

llms diffusion efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs

arXiv:2604. 23466v2 Announce Type: replace Abstract: NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs.

By Divakar Kumar Yadav, Tian Zhao, Deepak Kumar

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions

arXiv:2606. 05692v1 Announce Type: new Abstract: Deep learning has enabled significant advances in time-series causal inference, yet progress remains constrained by the lack of realistic benchmarks with observable counterfactual outcomes.

By Wenhao Mu, Facundo Yan, Anik Mumssen, Marisa Eisenberg, Alexander Rodr\'iguez

agents benchmarks

arXiv Machine Learning

Jun 5

Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Prediction

arXiv:2606. 05797v1 Announce Type: new Abstract: Longitudinal treatment decisions require predicting potential outcomes under future treatment sequences in the presence of time-varying confounding, heterogeneous patient dynamics, and limited domain-specific data.

By Amirhossein Zare, Amirhessam Zare, Herlock Rahimi, Reza Salarikia, Mohammad Kashkooli

benchmarks

arXiv Machine Learning

Jun 5

Learning Self-Correction in Vision-Language Models via Rollout Augmentation

arXiv:2602. 08503v2 Announce Type: replace-cross Abstract: Self-correction is essential for solving complex reasoning problems in vision-language models (VLMs).

By Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang

llms reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 5

GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

arXiv:2606. 05860v1 Announce Type: new Abstract: Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise.

By Oleeviya Babu Poikarayil, C\'edric Schockaert, Abdulrahman Nahhas, Christian Daase, Mursal Dawodi, Jawid Ahmad Baktash

llms agents efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents

arXiv:2606. 05296v1 Announce Type: new Abstract: LLM agents operate in two distinct regimes: open-weight agents amenable to reinforcement learning (RL) and black-box agents whose behaviour must be controlled purely at test time.

By Dae Yon Hwang, Raunaq Suri, Valentin Villecroze, Anthony L. Caterini, Jesse C. Cresswell, No\"el Vouitsis, Brendan Leigh Ross

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Unifying Dataset Pruning and Distillation for Efficient Large-scale Compression

arXiv:2502. 06434v2 Announce Type: replace-cross Abstract: Dataset pruning (DP) and dataset distillation (DD) fundamentally differ in their outputs: DP selects original image subsets, while DD generates synthetic images.

By Lingao Xiao, Songhua Liu, Yang He, Xinchao Wang

efficiency benchmarks

arXiv Machine Learning

Jun 5

EML-CD: Causal Mechanism Recovery via EML Symbolic Trees in Structure Learning

arXiv:2606. 05942v1 Announce Type: cross Abstract: Neural network (NN)-based nonlinear causal discovery methods recover DAG structure but leave each causal mechanism as a black box.

By Sota Asanuma

benchmarks

arXiv Machine Learning

Jun 5

A Differentiable Framework for Full and Phaseless Data Inversion Using Neural Implicit Contrast-Source Representation

arXiv:2508. 10555v2 Announce Type: replace-cross Abstract: In this study, we extend the contrast source inversion to a fully differentiable, unsupervised framework based on a neural implicit representation of the contrast source.

By Haoran Sun, Daoqi Liu, Hongyu Zhou, Maokun Li, Shenheng Xu, Fan Yang

benchmarks

arXiv Machine Learning

Jun 5

Tracing the Oracle: Improving Diffusion Timestep Scheduling for 3D CT Reconstruction

arXiv:2606. 06236v1 Announce Type: new Abstract: Pretrained diffusion models demonstrate impressive potential in solving highly ill-posed 3D computed tomography (CT) inverse problems, while the inference process suffers from significant computational overhead.

By Yujia Wu, Zhaoqiang Liu

diffusion benchmarks

arXiv Machine Learning

Jun 5

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Jun 5

The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models

arXiv:2606. 05169v1 Announce Type: new Abstract: We give a stereological theory of LLM benchmark coverage.

By Jason Z Wang

llms benchmarks

arXiv Machine Learning

Jun 5

Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search

arXiv:2606. 05729v1 Announce Type: cross Abstract: Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which is a combinatorial search problem that scales poorly with the number of random variables.

By Shing Yin Wong, Shaocheng Liu, Linqi Song, Amin Gohari, Cheuk Ting Li

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Step-adaptive multimodal fusion network with multi-scale cloud feature learning for ultra-short-term solar irradiance forecasting

arXiv:2606. 06102v1 Announce Type: cross Abstract: Ultra-short-term solar irradiance prediction is critical for photovoltaic system dispatch and power grid stability.

By Jingxin Zhang Xiaoqin Wang

multimodal benchmarks

arXiv Machine Learning

Jun 5

In-Context Multiple Instance Learning

arXiv:2606. 06458v1 Announce Type: new Abstract: Multiple Instance Learning (MIL) addresses problems where supervision is available at the level of bags of instances and has been successfully applied in fields ranging from computational pathology to satellite imagery.

By Alexander M\"ollers, Marvin Sextro, Julius Hense, Gabriel Dernbach, Klaus-Robert M\"uller

benchmarks

arXiv Machine Learning

Jun 5

Wall Shear Stress Reconstruction from Concentration: Differentiable Physics and Physics-Informed Neural Networks

arXiv:2606. 06313v1 Announce Type: cross Abstract: Wall shear stress (WSS) governs near-wall transport dynamics and is a key hemodynamic indicator in cardiovascular flows, yet remains difficult to infer accurately due to the need for precise computation of near-wall velocity gradients.

By Mahmoud Elhadidy, Siva Viknesh, Roshan M. D'Souza, Amirhossein Arzani

benchmarks

arXiv Machine Learning

Jun 5

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

arXiv:2606. 06314v1 Announce Type: cross Abstract: Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains.

By Anshima Singh, David J. Silvester

benchmarks

arXiv Machine Learning

Jun 5

HOPSE: Scalable Higher-Order Positional and Structural Encoder for Combinatorial Representations

arXiv:2505. 15405v3 Announce Type: replace Abstract: While Graph Neural Networks (GNNs) have proven highly effective at modeling relational data, pairwise connections cannot fully capture multi-way relationships naturally present in complex real-world systems.

By Guillermo Bern\'ardez, Marco Montagna, Louis Van Langendonck, Martin Carrasco, Amirreza Akbari, Louisa Cornelis, Mathilde Papillon, Pere Barlet-Ros, Nina Miolane, Lev Telyatnikov

benchmarks

arXiv Machine Learning

Jun 5

GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

arXiv:2606. 05441v1 Announce Type: new Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones.

By Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali, Gianfranco Doretto, Donald A. Adjeroh

nlp benchmarks

Get the daily briefing