Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Machine Learning

Sep 30

KernelOnet: An Interpretable Neural Operator Based on Kernel Functions

arXiv:2609.35938v1 Announce Type: new Abstract: This paper proposes an interpretable neural operator framework, the Kernel Operator Network (KernelOnet), which incorporates kernel functions explicitl...

By Yuan Guo, Hanshu Chen, Qiang Xi, Timon Rabczuk, Zhuojia Fu

benchmarks

arXiv Machine Learning

Sep 30

How Language Models Differ in Redistributing Attention-Head Activity Under Serial Demand

arXiv:2609.36221v1 Announce Type: new Abstract: The way a model distributes activity over each layer's attention heads offers a coarse view of how it routes information through depth; how this change...

By Johnny Jingze Li, Abdulla Kuleib, Kalyan Basu, Gabriel A. Silva

llms benchmarks

arXiv Machine Learning

Sep 30

GNA: Granular Neighbor Assembly for Retrieval-Augmented Multivariate Time-Series Forecasting

arXiv:2609.36281v1 Announce Type: new Abstract: Deep forecasters predict from a fixed-length lookback window, and lengthening it gives diminishing returns at a growing cost. Retrieval augmentation in...

By Vincent Uhse

llms rag benchmarks

arXiv AI

Sep 30

PyroStack: A Multi-Band Spatio-Temporal Sub-Daily Dataset for Wildfires in the United States

arXiv:2609.36315v1 Announce Type: cross Abstract: Wildfires are an increasing hazard to ecosystems, air quality, and human systems, creating a growing need for datasets that support systematic develo...

By Arya Kondur, Giosue Migliorini, Cameron Schmitt, Francesco Immorlano, Tairan Wang, Rebecca C. Scholten, Efi Foufoula-Georgiou, Gary Johnson, Chris Lautenberger, Valentin Waeselynck, J. Shane Romsos, Kasra Shamsaei, Alejandro Tejedor, Tianjia Liu, Yang Chen, Padhraic Smyth, James T. Randerson

benchmarks

arXiv Machine Learning

Sep 30

DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance

arXiv:2609.36472v1 Announce Type: new Abstract: Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian or ensembling surrogates for uncertainty quantification. While historically...

By Jonas Seng, Bennet Wittelsbach, Kristian Kersting

benchmarks

arXiv AI

Sep 30

HiTS-CL: A Continual Learning Framework for Long-Horizon Temporal Knowledge Graph Extrapolation

arXiv:2609.36559v1 Announce Type: cross Abstract: Extrapolative temporal knowledge graph reasoning (TKGR) predicts future facts from historical snapshots. Most existing methods train once on an early...

By Yansong Liu, Rui Liu, Yuan Zuo, Hongwei Zhao, Da Fu, Fuwei Zhang, Fuzhen Zhuang, Yong Chen, Zhe Li

fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 30

Sharp Convergence and Sampling Trade-offs for Riemannian Diffusion under Nonnegative Ricci Curvature

arXiv:2609.36568v1 Announce Type: new Abstract: Diffusion models have emerged as state-of-the-art generative models, with recent extensions from Euclidean spaces to Riemannian manifolds. However, exi...

By Yuhao Liu, Longbo Huang

diffusion benchmarks

arXiv Machine Learning

Sep 30

Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it

arXiv:2609.36612v1 Announce Type: new Abstract: Unlearning in large language models (LLMs) is typically evaluated at the output level, where a model appears to suppress sensitive or undesirable conte...

By Hadi Reisizadeh, Jiajun Ruan, Sijia Liu, Mingyi Hong

llms benchmarks safety

arXiv Machine Learning

Sep 30

PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation

arXiv:2609.36638v1 Announce Type: new Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable i...

By Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei, Liang Han

diffusion efficiency benchmarks

arXiv AI

Sep 30

Inducing Process Supervision from Outcome-Only Reinforcement Learning

arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....

By Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen, Yankai Lin

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

When Is Coarse Supervision Worth It? Cost-Aware Learning under Unknown Aggregation

arXiv:2609.36704v1 Announce Type: new Abstract: Modern learning systems often acquire supervision at multiple resolutions, trading annotation cost against information content. We study cost-aware two...

By Jianyu Xu, Smriti Jha, Aarti Singh, Bryan Wilder

benchmarks

arXiv AI

Sep 30

Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving

arXiv:2609.36750v1 Announce Type: cross Abstract: Self-rewarding reinforcement learning (RL) enables large language models (LLMs) to self-evolve without human labels. Existing ensemble-based methods...

By Yiming Wang, Yikang Liu, Qingyuan Tian, Xingyu Chen, Zhuosheng Zhang, Zhaopeng Tu, Rui Wang

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

Harnessing Large Language Models to Compile Task-Relevant Context into Bayesian Optimisation

arXiv:2609.36788v1 Announce Type: new Abstract: Incorporating rich task-relevant context, such as domain knowledge and external observations, is a key capability yet remains challenging for Bayesian...

By Zhongwei Yu, Sourabh Roy, Bin Cao, Xue Yan, Anjie Liu, Jun Wang

llms rag benchmarks

arXiv Machine Learning

Sep 30

Towards Better Training Signal: Advantage Clipped Policy Optimization

arXiv:2609.36816v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a cornerstone for improving the reasoning capabilities of large language models (LLMs), but the need for on-poli...

By Ruichuan Huang, Jinghan Liu, Congliang Chen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

An Effective, Reliable, and Robust Framework for Human Activity Recognition Using Wearable Sensors

arXiv:2609.36848v1 Announce Type: new Abstract: Human Activity Recognition (HAR) through wearable sensors greatly improves the quality of human life through its multiple applications. For HAR, multi-...

By Nafees Ahmad, Ho-fung Leung, Muhammad Adil Abid, Sadia Shakil

benchmarks

arXiv Machine Learning

Sep 30

TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Tasks

arXiv:2609.36968v1 Announce Type: new Abstract: Unsupervised tabular anomaly detection (TAD) aims to identify anomalous rows in tabular data using normal training samples. While conventional methods...

By Doyun Choi, Dooho Lee, Jaemin Yoo

llms benchmarks

arXiv Machine Learning

Sep 30

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

arXiv:2609.37114v1 Announce Type: new Abstract: DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches s...

By Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung

benchmarks

arXiv Machine Learning

Sep 30

Corruption-Robust Sparse Linear Contextual Bandits with Knapsack Constraints

arXiv:2609.37189v1 Announce Type: new Abstract: We study sparse linear contextual bandits with knapsack constraints under joint reward and consumption corruption. Consumption corruption creates a cha...

By Yige Wang, Hanyang Li, Yiming Zong, Wanteng Ma, Jiashuo Jiang

benchmarks safety

arXiv Machine Learning

Sep 30

MoTIF-X: A Multimodal Tokenized Framework for Interpretable and Extensible Molecular Representation Learning

arXiv:2609.37384v1 Announce Type: new Abstract: Molecular representation learning is central to computer-aided drug discovery. Molecular graphs, SMILES strings, and 3D conformations provide complemen...

By Linqing Mo, Jiayu Zhou, Bin Chen

fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Sep 30

Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces

arXiv:2609.37470v1 Announce Type: new Abstract: A probability used for a decision should refer to the same event across equivalent requests. We introduce probability contracts, a benchmark connecting...

By Han Chen, Yingrui Li

llms benchmarks

Get the daily briefing