Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,090 stories · RSS feed

arXiv Machine Learning

Jun 5

Differentiable Efficient Operator Search

arXiv:2606. 05232v1 Announce Type: new Abstract: Efficient multimodal foundation models often rely on manually designed token-reduction operators, such as pruning, merging, pooling, and adaptive reweighting.

By Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo, Weiguo Feng, Tao Huang, Cho-Jui Hsieh, Chang Xu

efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

Latent Reasoning with Normalizing Flows

arXiv:2606. 06447v1 Announce Type: cross Abstract: Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation.

By Guancheng Tu, Xiangjun Fu, Suhao Yu, Yao Tang, Haoqiang Kang, Lianhui Qin, Yizhe Zhang, Jiatao Gu

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Reflex: Reinforcement Learning with Reflection Symmetry Exploitation in State-Based Continuous Control

arXiv:2605. 23415v2 Announce Type: replace Abstract: Reinforcement learning has long struggled with poor sample efficiency.

By Shuai Zhen, Yifan Zhang, Yuling Wang, Yanhua Yu

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 5

Addressing Imbalance in Multi-Label Data via Label-Specific Distance-based Oversampling

arXiv:2606. 05927v1 Announce Type: new Abstract: The complex imbalanced label distribution poses a crucial challenge to multi-label classification, as most classifiers are biased towards the majority class and high-frequent labels.

By Bin Liu, Jun Wu, Haoyu Peng, Ao Zhou, Jin Wang, QiaoSong Chen, Grigorios Tsoumakas

benchmarks

arXiv Machine Learning

Jun 5

Q-GNN: Query-Conditioned Graph Neural Networks with Type Awareness for Knowledge Graph Completion

arXiv:2606. 05639v1 Announce Type: new Abstract: Knowledge Graph Completion (KGC) aims at predicting missing triplets from incomplete knowledge graphs, which is crucial for downstream applications.

By Dongxiao He, Ruqiong Zhang, Zhizhi Yu, Ling Ding, Di Jin, Guangquan Xu, Zhiyong Feng

llms benchmarks

arXiv Machine Learning

Jun 5

HyFAD: Hybrid Time-Frequency Diffusion with Frequency-Aware Embedding for Time Series Imputation

arXiv:2606. 05239v1 Announce Type: cross Abstract: Diffusion models have demonstrated strong performance in time series modeling due to their ability to progressively capture complex data distributions through iterative denoising.

By Hongfan Gao, Wangmeng Shen, Bin Yang, Jilin Hu

rag diffusion benchmarks

arXiv Machine Learning

Jun 5

Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation

arXiv:2606. 05988v1 Announce Type: new Abstract: Reasoning models produce long chain-of-thought traces that are costly to distill and encourage verbose student outputs.

By Maxime Griot, Paul Steven Scotti, Tanishq Mathew Abraham

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 5

Self-Augmenting Retrieval for Diffusion Language Models

arXiv:2606. 06474v1 Announce Type: cross Abstract: Discrete diffusion language models generate text by iteratively denoising an entire response in parallel.

By Paul J\"unger, Justin Lovelace, Linxi Zhao, Dongyoung Go, Kilian Q. Weinberger

llms rag diffusion efficiency benchmarks

arXiv Machine Learning

Jun 5

Adaptive Oscillatory-State Alignment for Time Series Forecasting

arXiv:2606. 06010v1 Announce Type: new Abstract: Long-term time series forecasting benefits from inductive biases that expose recurring temporal structure.

By Zhangyao Song, Ziqiong Li, Xiangfei Qiu, Chao Zha, Yinfei Xu, Tao Guo

efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Geodesic Semantic Search: Cartographic Navigation of Citation Graphs with Learned Local Riemannian Maps

arXiv:2602. 23665v5 Announce Type: replace-cross Abstract: We present Geodesic Semantic Search (GSS), a retrieval system that learns node-specific Riemannian metrics on citation graphs to enable geometry-aware semantic search.

By Brandon Yee, Lucas Wang, Kundana Kommini

rag benchmarks

arXiv Machine Learning

Jun 5

Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series

arXiv:2506. 00188v2 Announce Type: replace Abstract: Early and accurate detection of anomalies in time-series data is critical due to the substantial risks associated with false or missed detections.

By Md Mahmuddun Nabi Murad, Yasin Yilmaz

rag benchmarks

arXiv Machine Learning

Jun 5

Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction

arXiv:2606. 06303v1 Announce Type: new Abstract: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining.

By Hongkun Dou, Zike Chen, Fengji Li, Hongjue Li, Yue Deng

diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Prediction

arXiv:2606. 05797v1 Announce Type: new Abstract: Longitudinal treatment decisions require predicting potential outcomes under future treatment sequences in the presence of time-varying confounding, heterogeneous patient dynamics, and limited domain-specific data.

By Amirhossein Zare, Amirhessam Zare, Herlock Rahimi, Reza Salarikia, Mohammad Kashkooli

benchmarks

arXiv Machine Learning

Jun 5

The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models

arXiv:2606. 05169v1 Announce Type: new Abstract: We give a stereological theory of LLM benchmark coverage.

By Jason Z Wang

llms benchmarks

arXiv Machine Learning

Jun 5

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Jun 5

Tracing the Oracle: Improving Diffusion Timestep Scheduling for 3D CT Reconstruction

arXiv:2606. 06236v1 Announce Type: new Abstract: Pretrained diffusion models demonstrate impressive potential in solving highly ill-posed 3D computed tomography (CT) inverse problems, while the inference process suffers from significant computational overhead.

By Yujia Wu, Zhaoqiang Liu

diffusion benchmarks

arXiv Machine Learning

Jun 5

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

arXiv:2606. 06314v1 Announce Type: cross Abstract: Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains.

By Anshima Singh, David J. Silvester

benchmarks

arXiv Machine Learning

Jun 5

Wall Shear Stress Reconstruction from Concentration: Differentiable Physics and Physics-Informed Neural Networks

arXiv:2606. 06313v1 Announce Type: cross Abstract: Wall shear stress (WSS) governs near-wall transport dynamics and is a key hemodynamic indicator in cardiovascular flows, yet remains difficult to infer accurately due to the need for precise computation of near-wall velocity gradients.

By Mahmoud Elhadidy, Siva Viknesh, Roshan M. D'Souza, Amirhossein Arzani

benchmarks

arXiv Machine Learning

Jun 5

Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search

arXiv:2606. 05729v1 Announce Type: cross Abstract: Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which is a combinatorial search problem that scales poorly with the number of random variables.

By Shing Yin Wong, Shaocheng Liu, Linqi Song, Amin Gohari, Cheuk Ting Li

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation

arXiv:2605. 31278v2 Announce Type: replace-cross Abstract: Reliable evaluation of agentic systems requires unbiased estimates with valid uncertainty, but standard practice navigates between costly human annotation and biased LLM-as-judge proxies.

By Gr\'egoire Martinon, Ibrahim Merad, Mohammed Raki

llms agents benchmarks

Get the daily briefing