Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Machine Learning

Sep 30

PHASE: Multi-Regime Modeling of Incompressible Magnetohydrodynamics

arXiv:2609.37609v1 Announce Type: new Abstract: Magnetohydrodynamics (MHD) is central to plasma modeling in astrophysics, space science, fusion, and engineering, but resolving multiscale MHD dynamics...

By Radhika Achikanath Chirakkara, Rajdeep Haldar, Zezheng Song, Jiequn Han

fine-tuning benchmarks

arXiv Machine Learning

Sep 30

LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling

arXiv:2609.37675v1 Announce Type: new Abstract: Protein Language Models (PLMs) have made remarkable progress following scaling laws established in natural language processing across sequence- and str...

By Biswajit Banerjee, Claudia Alvarez Carreno, Anton S. Petrov

llms rag nlp benchmarks

arXiv Machine Learning

Sep 30

Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

arXiv:2609.37924v1 Announce Type: new Abstract: Recent work on anchored diffusion language models improves denoising by shaping an intermediate latent space with supervised important-token targets. I...

By Joel Anto Paul, Litu Rout, Aditya Akella, Sanjay Shakkottai

llms diffusion efficiency benchmarks

arXiv Machine Learning

Sep 30

Alpha Diffusion Language Models: Factorization Alone Is Not the Problem

arXiv:2609.38066v1 Announce Type: new Abstract: Discrete diffusion language models can generate multiple tokens in parallel, but reducing the number of denoising steps can lead to inconsistent predic...

By Nikita Gushchin, Dmitry Baranchuk, Alexander Korotin

llms diffusion benchmarks

arXiv Machine Learning

Sep 30

Tail-Influence Sampling for CVaR Policy Evaluation

arXiv:2609.38096v1 Announce Type: new Abstract: Policies with similar mean returns can differ sharply in rare failures, yet estimating lower-tail conditional value-at-risk (CVaR) accurately can requi...

By Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer, Rasul Tutunov, Haitham Bou-Ammar

benchmarks

arXiv Machine Learning

Sep 30

CI-PINN: Causal Integral Physics-Informed Neural Network for Solving Evolution Equations

arXiv:2609.36615v1 Announce Type: cross Abstract: Physics-informed neural networks (PINNs) solve partial differential equations (PDEs) by incorporating governing physical laws into the training loss....

By Xiaodong Feng, Ziyu Sun, Tao Tang, Xiaoliang Wan, Tao Zhou

benchmarks

arXiv Machine Learning

Sep 30

AutoMark: Enabling Autoresearch to Discover Better LLM Watermarks

arXiv:2609.37310v1 Announce Type: cross Abstract: With LLM watermarking being deployed commercially and now required by regulations, improving its reliability and effectiveness has become crucial. Ye...

By Thibaud Gloaguen, Robin Staab, Martin Vechev

llms agents benchmarks

arXiv Machine Learning

Sep 30

Your Benchmark Is Not Saturated: Reviving Multiple-Choice Evaluation with Answer Pooling

arXiv:2609.37494v1 Announce Type: cross Abstract: Multiple-choice benchmarks are cheap to grade and are running out of room, and the standard remedy, writing harder items, is slow and repeated for ev...

By Mohamed Eltahir, Abobaker Ahmed, Nawaf Barebood, Hussain Bu Subayt, Tanveer Hussain, Naeemullah Khan

benchmarks

arXiv Machine Learning

Sep 30

PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs

arXiv:2511.22435v2 Announce Type: replace Abstract: Invariant learning on graphs aims to build predictors that rely on causal substructures rather than on environment-specific shortcuts. Current meth...

By Ali Ghasemi, Farooq Ahmad Wani, Maria Sofia Bucarelli, Fabrizio Silvestri

benchmarks

arXiv Machine Learning

Sep 30

Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression

arXiv:2602.02848v2 Announce Type: replace Abstract: Advances in large language models have driven strong performance across many tasks, but their memory and compute costs still hinder deployment. SVD...

By Ali Abbasi, Chayne Thrash, Haoran Qin, Shansita Sharma, Sepehr Seifi, Soheil Kolouri

llms benchmarks

arXiv AI

Sep 30

Engineering Efficient Self-Play Chess: Search, Replay, and Throughput Under Limited Compute

arXiv:2609.37447v1 Announce Type: cross Abstract: How strong can an AlphaZero-style chess system become under limited training compute when its entire learning loop is engineered for efficiency? We t...

By Bertil Braun

reinforcement-learning efficiency benchmarks

arXiv AI

Sep 30

TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning

arXiv:2609.37581v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visu...

By Jing Wang, Zhiping Wu, Dongdong Ren, Youfang Han, Wei Zhao, Wenbin Li

llms efficiency multimodal benchmarks

arXiv Machine Learning

Sep 30

Formalizing the Sampling Design Space of Diffusion-Based Generative Models via Adaptive Solvers and Wasserstein-Bounded Timesteps

arXiv:2602.12624v2 Announce Type: replace Abstract: Diffusion-based generative models have achieved remarkable performance across various domains, yet their practical deployment is often limited by h...

By Sangwoo Jo, Sungjoon Choi

diffusion benchmarks