Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Machine Learning

Sep 22

CLEAR: Complex Learned Explicit Analytical Regularization for Ultra-Accelerated 4D Flow CMR Reconstruction

arXiv:2609.22950v1 Announce Type: cross Abstract: While compressed-sensing regularizers enable interpretable reconstruction of 4D Flow CMR through transparent variational objectives, their hand-craft...

By German Sh\^ama Wache, Sebastian Neumayer

benchmarks safety

arXiv Machine Learning

Sep 22

R-GEAN: Regimen-Guided Edit Action Network for Within-Admission Medication Change Prediction

arXiv:2609.22959v1 Announce Type: cross Abstract: The medications prescribed to a patient often change during a hospital admission as clinicians start, stop, or continue therapies. We study whether m...

By Regan Mahat, Mansu Kim

benchmarks

arXiv Machine Learning

Sep 22

Signal-Informed Temporal Routing for Vinyl Defect Regime Detection

arXiv:2609.23164v1 Announce Type: cross Abstract: Vinyl restoration systems must distinguish isolated clicks, short bursts, dense crackle, and overlapping damage before selecting a repair operation....

By Yi-Hung Kan, Homayoon Beigi

benchmarks

arXiv Machine Learning

Sep 22

SDC-GON: Singular Decomposition and Consistency-Regularized Green's Operator Networks for Solving Partial Differential Equations

arXiv:2609.23206v1 Announce Type: cross Abstract: Green's function based operator approximation offers an efficient route for solving linear partial differential equations under varying boundary cond...

By Yingchao Huang, Xin Wang, Shanshan Yao, Fanhua Zeng, Wei Peng

diffusion benchmarks

arXiv Machine Learning

Sep 22

Density-Ratio Rescoring for Imbalanced Classification

arXiv:2609.23926v1 Announce Type: cross Abstract: Density-Ratio Rescoring (DRR) augments a classifier trained at the original class prior with a survey-raking dual score. Raking reweights the majorit...

By Dongha Kim, Seunghwan Park

benchmarks

arXiv Machine Learning

Sep 22

The Operational Value of Spatial Dependence in Renewable Forecast Scenarios for Single-Period Economic Dispatch: A Controlled Ablation Study

arXiv:2609.23988v1 Announce Type: cross Abstract: Renewable forecasts are evaluated by statistical skill (e.g., CRPS), but grid operators pay for realized dispatch cost. We diagnose what drives dispa...

By Jayakumar Manoharan

benchmarks

arXiv Machine Learning

Sep 22

Causal Bayesian Optimization: Foundations, Methods, and Applications

arXiv:2609.24112v1 Announce Type: cross Abstract: Causal Bayesian Optimization (CBO) combines causal inference with Bayesian optimization to enable sample-efficient intervention selection in systems...

By Chenfeng Huang, Thuy T. Le, Zixuan Ma, Hien Tran

benchmarks

arXiv Machine Learning

Sep 22

MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents

arXiv:2609.24161v1 Announce Type: cross Abstract: As LLM agents increasingly interact with external tools through standardized protocols such as MCP, tool-interface design becomes a critical yet unde...

By Demetris Paschalides, Moysis Symeonides, George Pallis, Marios D. Dikaiakos

llms agents benchmarks

arXiv Machine Learning

Sep 22

Beyond Point Prediction: Artificial Representative Trees with Uncertainty

arXiv:2609.24528v1 Announce Type: cross Abstract: Random forests (RFs) predict well but are opaque, whereas single decision trees are interpretable but unstable. Artificial representative trees (ARTs...

By Lea L. Mairh\"ofer, Silke Szymczak, Bj\"orn-Hergen Laabs, Tuwe L\"ofstr\"om-Cavallin

benchmarks safety

arXiv Machine Learning

Sep 22

onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

arXiv:2609.24983v1 Announce Type: cross Abstract: We present onPanda, an interactive tool for efficiently annotating LLM alignment data and agent trajectories. onPanda adopts token-level correction a...

By Lei Yang, Mengyin Liu, Jia Wang, Hangyu Guo, Liang Zhao, Zheng Ge, Kang An, Binxing Jiao, Qi Han, Daxin Jiang, Siqi Shen, Xiangyu Zhang

llms agents fine-tuning benchmarks safety

arXiv Machine Learning

Sep 22

Disassociating performance from compositional feature learning

arXiv:2505.09716v4 Announce Type: replace Abstract: Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and...

By George Dimitriadis, Spyridon Samothrakis

llms benchmarks safety

arXiv Machine Learning

Sep 22

Glass-Box Deep Learning for FDIA Detection in Nonlinear Automatic Generation Control: A Kolmogorov-Arnold Network Approach

arXiv:2509.05259v2 Announce Type: replace Abstract: Automatic Generation Control (AGC) plays a critical role in maintaining power balance across multi-area power systems. However, its complete relian...

By Ahmad Mohammad Saber, Alok Paranjape, Jehad Jilan, Niranjana Naveen Nambiar, Amr Youssef, Deepa Kundur

fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Sep 22

Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization

arXiv:2512.07419v3 Announce Type: replace Abstract: Mixed-Precision Quantization (MPQ) liberates Deep Neural Networks (DNNs) from the Out-Of-Memory (OOM) bottleneck and has garnered increasing resear...

By Haidong Kang, Jun Du, Guo Yu

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 22

Vector-Valued Distributional Reinforcement Learning Policy Evaluation: A Hilbert Space Embedding Approach

arXiv:2601.18952v3 Announce Type: replace Abstract: We propose Kernel Embedding Distributional Reinforcement Learning (KE-DRL) an offline method to estimate kernel mean embeddings of conditional mult...

By Mehrdad Mohammadi, Qi Zheng, Ruoqing Zhu

rag reinforcement-learning benchmarks

arXiv Machine Learning

Sep 22

Holographic generative flows with AdS/CFT

arXiv:2601.22033v2 Announce Type: replace Abstract: Holography, in the form of the anti-de Sitter/conformal field theory (AdS/CFT) correspondence, offers a natural setting for generative modelling. D...

By Ehsan Mirafzali, Sanjit Shashi, Sanya Murdeshwar, Edgar Shaghoulian, Daniele Venturi, Razvan Marinescu

diffusion benchmarks

arXiv Machine Learning

Sep 22

Focused PU learning from imbalanced data

arXiv:2605.14467v2 Announce Type: replace Abstract: We propose a new method of learning from positive and unlabeled (PU) examples in highly imbalanced datasets. Many real-world problems, such as dise...

By Elias Zavitsanos, Georgios Paliouras

benchmarks

arXiv Machine Learning

Sep 22

Reforge: Low-Latency Distributed GNN Serving with Selective Embedding Recomputation

arXiv:2501.08547v2 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) have been widely adopted for their ability to compute expressive node representations in graph datasets. However...

By Geon-Woo Kim, Donghyun Kim, Jeongyoon Moon, Henry Liu, Tarannum Khan, Anand Iyer, Daehyeok Kim, Aditya Akella

rag benchmarks

arXiv Machine Learning

Sep 22

Plan-Driven Adaptive Bidding for First-Price Auctions with Budget Constraints under Nonstationarity

arXiv:2505.02796v3 Announce Type: replace-cross Abstract: We study budget pacing in repeated first-price auctions when an advertiser's private-value distributions change over time and the stationary...

By Yige Wang, Jiashuo Jiang

benchmarks

arXiv Machine Learning

Sep 22

MMS-VPR: A Fine-Grained Multimodal Street-Level Visual Place Recognition Dataset and Evaluation Benchmark for Dense Pedestrian Environments

arXiv:2505.12254v3 Announce Type: replace-cross Abstract: Existing visual place recognition (VPR) datasets predominantly rely on vehicle-mounted imagery, offer limited multimodal diversity, and under...

By Yiwei Ou, Xiaobin Ren, Ronggui Sun, Guansong Gao, Kaiqi Zhao, Manfredo Manfredini

llms multimodal benchmarks

arXiv Machine Learning

Sep 22

TempCore: Are Video QA Benchmarks Temporally Grounded?

arXiv:2509.01167v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) can ingest only a limited number of video frames, making frame selection a practical necessity. But do current...

By Hyunjong Ok, Jaeho Lee

llms multimodal benchmarks

Get the daily briefing