Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 5

Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series

arXiv:2506. 00188v2 Announce Type: replace Abstract: Early and accurate detection of anomalies in time-series data is critical due to the substantial risks associated with false or missed detections.

By Md Mahmuddun Nabi Murad, Yasin Yilmaz

rag benchmarks

arXiv Machine Learning

Jun 5

Geodesic Semantic Search: Cartographic Navigation of Citation Graphs with Learned Local Riemannian Maps

arXiv:2602. 23665v5 Announce Type: replace-cross Abstract: We present Geodesic Semantic Search (GSS), a retrieval system that learns node-specific Riemannian metrics on citation graphs to enable geometry-aware semantic search.

By Brandon Yee, Lucas Wang, Kundana Kommini

rag benchmarks

arXiv Machine Learning

Jun 5

Addressing Imbalance in Multi-Label Data via Label-Specific Distance-based Oversampling

arXiv:2606. 05927v1 Announce Type: new Abstract: The complex imbalanced label distribution poses a crucial challenge to multi-label classification, as most classifiers are biased towards the majority class and high-frequent labels.

By Bin Liu, Jun Wu, Haoyu Peng, Ao Zhou, Jin Wang, QiaoSong Chen, Grigorios Tsoumakas

benchmarks

arXiv Machine Learning

Jun 5

Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search

arXiv:2606. 05729v1 Announce Type: cross Abstract: Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which is a combinatorial search problem that scales poorly with the number of random variables.

By Shing Yin Wong, Shaocheng Liu, Linqi Song, Amin Gohari, Cheuk Ting Li

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

MSTN: A Lightweight and Fast Model for General TimeSeries Analysis

arXiv:2511. 20577v5 Announce Type: replace Abstract: Real-world time series often exhibit strong non-stationarity, complex nonlinear dynamics, and behavior expressed across multiple temporal scales, from rapid local fluctuations to slow-evolving long-range trends.

By Sumit S Shevtekar, Chandresh K Maurya

llms nlp benchmarks

arXiv Machine Learning

Jun 5

PF$\Delta$: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations

arXiv:2510. 22048v4 Announce Type: replace Abstract: Power flow (PF) calculations are the backbone of real-time grid operations, across workflows such as contingency analysis (where repeated PF evaluations assess grid security under outages) and topology optimization (which involves PF-based searches over combinatorially large action spaces).

By Ana K. Rivera, Anvita Bhagavathula, Alvaro Carbonero, Priya Donti

benchmarks

arXiv Machine Learning

Jun 5

HOPSE: Scalable Higher-Order Positional and Structural Encoder for Combinatorial Representations

arXiv:2505. 15405v3 Announce Type: replace Abstract: While Graph Neural Networks (GNNs) have proven highly effective at modeling relational data, pairwise connections cannot fully capture multi-way relationships naturally present in complex real-world systems.

By Guillermo Bern\'ardez, Marco Montagna, Louis Van Langendonck, Martin Carrasco, Amirreza Akbari, Louisa Cornelis, Mathilde Papillon, Pere Barlet-Ros, Nina Miolane, Lev Telyatnikov

benchmarks

arXiv Machine Learning

Jun 5

Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorization

arXiv:2310. 04649v3 Announce Type: replace Abstract: We introduce NPEFF (Non-Negative Per-Example Fisher Factorization), an interpretability method that aims to uncover strategies used by a model to generate its predictions.

By Michael Matena, Colin Raffel

llms benchmarks safety

arXiv Machine Learning

Jun 5

The Post-GCN Decade Revisited: Curvature-Stratified Evaluation of Relational Learning

arXiv:2606. 06397v1 Announce Type: new Abstract: Current evaluation practices in relational learning rely heavily on flat leaderboards that average performance across heterogeneous datasets, implicitly assuming a uniform underlying structure.

By Shuo Wang, Xiangyu Wang, Quanxin Wang, Bailin Wu, Bokui Wang, Shunyang Huang, Boyan Deng, Haonan Liu, Ruiyi Fang, Zhenxiang Xu, Boyu Wang, Zhao Kang

benchmarks safety

arXiv Machine Learning

Jun 5

Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning

arXiv:2606. 05675v1 Announce Type: new Abstract: Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge.

By Hongye Xu, Bartosz Krawczyk

rag benchmarks safety

arXiv Machine Learning

Jun 5

An accurate nucleic acid-small molecule docking framework via geometric deep learning with large-scale pretraining

arXiv:2606. 05198v1 Announce Type: cross Abstract: Nucleic acids are increasingly recognized as therapeutic targets beyond conventional protein-centered drug discovery, yet accurate and efficient docking of small molecules to nucleic acid structures remains challenging.

By Shi Li (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China), Xujun Zhang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China), Mingquan Liu (Faculty of Health Sciences, University of Macau, Macau SAR, China), Hui Zhang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Shuoying Jia (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Yu Kang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Tingjun Hou (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Zhejiang Provincial Key Laboratory for Intelligent Drug Discovery and Development, Jinhua Institute of Zhejiang University, Zhejiang, China), Peichen Pan (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Zhejiang Provincial Key Laboratory for Intelligent Drug Discovery and Development, Jinhua Institute of Zhejiang University, Zhejiang, China)

fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

arXiv:2606. 05516v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distributed across layers.

By Wanhao Yu, Ziyan Wang, Zheng Wang, Abeer Matar Almalky, Yihang Zuo, Shuteng Niu, Sen Lin, Adnan Siraj Rakin, Deliang Fan, Li Yang

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs

arXiv:2606. 05376v1 Announce Type: new Abstract: Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators.

By Jingyao Wu, Ashley Wang, Keane Ong, Paul Pu Liang, Rosalind Picard

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 5

ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

arXiv:2606. 05170v1 Announce Type: new Abstract: At matched accuracy, open-weight LLMs differ substantially in the shape of their error severity distribution -- a difference invisible to the scalar error rate.

By Jason Z Wang

llms benchmarks safety

arXiv Machine Learning

Jun 5

Wall Shear Stress Reconstruction from Concentration: Differentiable Physics and Physics-Informed Neural Networks

arXiv:2606. 06313v1 Announce Type: cross Abstract: Wall shear stress (WSS) governs near-wall transport dynamics and is a key hemodynamic indicator in cardiovascular flows, yet remains difficult to infer accurately due to the need for precise computation of near-wall velocity gradients.

By Mahmoud Elhadidy, Siva Viknesh, Roshan M. D'Souza, Amirhossein Arzani

benchmarks

arXiv Machine Learning

Jun 5

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

arXiv:2606. 06314v1 Announce Type: cross Abstract: Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains.

By Anshima Singh, David J. Silvester

benchmarks

arXiv Machine Learning

Jun 5

Tracing the Oracle: Improving Diffusion Timestep Scheduling for 3D CT Reconstruction

arXiv:2606. 06236v1 Announce Type: new Abstract: Pretrained diffusion models demonstrate impressive potential in solving highly ill-posed 3D computed tomography (CT) inverse problems, while the inference process suffers from significant computational overhead.

By Yujia Wu, Zhaoqiang Liu

diffusion benchmarks

arXiv Machine Learning

Jun 5

GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

arXiv:2606. 05441v1 Announce Type: new Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones.

By Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali, Gianfranco Doretto, Donald A. Adjeroh

nlp benchmarks

arXiv Machine Learning

Jun 5

Scalable Reinforcement Learning via Adaptive Batch Scaling

arXiv:2605. 21557v2 Announce Type: replace-cross Abstract: Conventional wisdom holds that large-batch training is fundamentally incompatible with Reinforcement Learning (RL) - beyond a modest threshold, increasing batch sizes typically yields diminishing returns or performance degradation due to the inherent non-stationarity of the data distribution.

By Jongchan Park

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.

By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal

llms diffusion benchmarks safety

Get the daily briefing