Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 5

Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval

arXiv:2606. 05415v1 Announce Type: cross Abstract: Real-world data spans tables, documents, and semi-structured files with implicit semantics.

By Padmaja Jonnalagedda, Yuguang Yao, Xiang Gao, Hilaf Hasson, Kamalika Das

llms rag agents benchmarks

arXiv Machine Learning

Jun 5

Multimarginal flow matching with optimal transport potentials

arXiv:2606. 05327v1 Announce Type: new Abstract: Flow matching (FM) has emerged as a powerful framework for learning dynamic transport maps between two empirical distributions.

By Raghav Kansal, David Crair, Nghia Nguyen, Scott Pope, Bradley Parry

diffusion benchmarks

arXiv Machine Learning

Jun 5

Query-efficient model evaluation using cached responses

arXiv:2605. 07096v2 Announce Type: replace Abstract: Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment.

By Hayden Helm, Ben Johnson, Carey Priebe

benchmarks

arXiv Machine Learning

Jun 5

Reframing preprocessing selection as model-internal calibration in near-infrared spectroscopy: A large-scale benchmark of operator-adaptive PLS and Ridge models

arXiv:2605. 13587v3 Announce Type: replace-cross Abstract: Preprocessing screening is often the most expensive part of a near-infrared spectroscopy calibration workflow.

By Gregory Beurier (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Robin Reiter (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Camille No\^us (Laboratoire Cogitamus), Lauriane Rouan (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France), Denis Cornet (CIRAD, UMR AGAP Institut, Montpellier, France, UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France)

benchmarks

arXiv Machine Learning

Jun 5

A New Quaternion-Joint Cable-Driven Redundant Manipulator Configuration and its Control Through FABRIK and Residual Reinforcement Learning

arXiv:2606. 05236v1 Announce Type: cross Abstract: Robotic arms capable of traversing arbitrary spatial paths, especially in highly obstructed workspaces, are highly desired across several industries.

By Tanapath Pornthisan, Thanapat Kemthong, Thanyapisit Kangsathien, Pasut Aranchaiya, Paulo Garcia, Viboon Sangveraphunsiri

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 5

Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction

arXiv:2606. 06303v1 Announce Type: new Abstract: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining.

By Hongkun Dou, Zike Chen, Fengji Li, Hongjue Li, Yue Deng

diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning

arXiv:2512. 09706v2 Announce Type: replace Abstract: The paradigm of agentic AI is shifting from engineered complex workflows to post-training native models.

By Kaichen He, Zihao Wang, Muyao Li, Anji Liu, Yitao Liang

agents reinforcement-learning robotics fine-tuning benchmarks

arXiv Machine Learning

Jun 5

GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

arXiv:2606. 05441v1 Announce Type: new Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones.

By Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali, Gianfranco Doretto, Donald A. Adjeroh

nlp benchmarks

arXiv Machine Learning

Jun 5

FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting

arXiv:2408. 11336v3 Announce Type: replace Abstract: Climate change stands as one of the most pressing global challenges of the twenty-first century, with far-reaching consequences such as rising sea levels, melting glaciers, and increasingly extreme weather patterns.

By Tajamul Ashraf, Janibul Bashir

llms benchmarks safety

arXiv Machine Learning

Jun 5

The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models

arXiv:2606. 05169v1 Announce Type: new Abstract: We give a stereological theory of LLM benchmark coverage.

By Jason Z Wang

llms benchmarks

arXiv Machine Learning

Jun 5

Representing Research Attention as Contextually Structured Flows

arXiv:2606. 05895v1 Announce Type: cross Abstract: Research attention is widely used as an indicator of visibility, influence, and societal uptake, yet it is typically represented as aggregated counts that do not preserve how attention develops across contexts over time.

By Jessica Rodrigues, Angelo Salatino, Gard Jenset, Scott Hale

benchmarks

arXiv Machine Learning

Jun 5

GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

arXiv:2606. 05860v1 Announce Type: new Abstract: Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise.

By Oleeviya Babu Poikarayil, C\'edric Schockaert, Abdulrahman Nahhas, Christian Daase, Mursal Dawodi, Jawid Ahmad Baktash

llms agents efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Know Yourself Better: Diverse Object-Related Features Improve Open Set Recognition

arXiv:2404. 10370v4 Announce Type: replace-cross Abstract: Open set recognition (OSR) is a critical aspect of machine learning, addressing the challenge of detecting novel classes during inference.

By Jiawen Xu, Margret Keuper

benchmarks

arXiv Machine Learning

Jun 5

Towards Label-Noise Resistant Learning via Optimal Brain Damage Masking

arXiv:2508. 09697v3 Announce Type: replace Abstract: Noisy labels are inevitable in real-world scenarios.

By Xinlei Zhang, Fan Liu, Chuanyi Zhang, Fan Cheng, Qian Li, Yuhui Zheng

efficiency benchmarks

arXiv Machine Learning

Jun 5

Performance Evaluation of GraphCast for Medium-Range Weather Forecasting over Brazil

arXiv:2606. 06348v1 Announce Type: new Abstract: The paradigm of global weather forecasting is rapidly shifting with the emergence of Machine Learning Weather Prediction models (MLWP).

By Wolfgang R. Rowell Jr., Lucas S. Kupssinsk\"u

benchmarks

arXiv Machine Learning

Jun 5

The Equilibrium Response of Atmospheric Machine-Learning Models to Uniform Sea Surface Temperature Warming

arXiv:2510. 02415v3 Announce Type: replace-cross Abstract: Machine learning models for the global atmosphere that are capable of producing stable, multi-year simulations of Earth's climate have recently been developed.

By Bosong Zhang, Timothy M. Merlis

benchmarks

arXiv Machine Learning

Jun 5

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Jun 5

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

arXiv:2505. 05026v5 Announce Type: replace-cross Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept.

By Jaehyun Jeon, Min Soo Kim, Jang Han Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, Youngjae Yu

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Localizing Prompt Ambiguity in Large Language Models with Probe-Targeted Attribution

arXiv:2606. 05486v1 Announce Type: cross Abstract: Prompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens.

By Govind Ramesh, Yao Dou, Wei Xu

llms benchmarks

Get the daily briefing