Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning

Oct 7

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

arXiv:2610.07967v1 Announce Type: new Abstract: As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their r...

By Yiming Xu, Hongyue Yu, Beihua Yang, Zihan Chen, Yixin Liu, Zhen Peng, Bin Shi, Bo Dong, Chao Shen, Irwin King, Qinghua Zheng

llms agents benchmarks

arXiv Machine Learning

Oct 7

Spectra: Exact Component Transport for Test-Time Prior Adaptation in Simulation-Based Inference

arXiv:2610.08021v1 Announce Type: new Abstract: Simulation-based inference (SBI) has become a powerful approach to Bayesian inference in complex scientific models whose likelihoods are difficult or i...

By Xin Zhao, Nico Scherf, Robert Trampel, Kerrin J. Pine, Nikolaus Weiskopf

diffusion benchmarks

arXiv Machine Learning

Oct 7

Evolutionary One-Step Generators: Fast and Diverse Sampling for Discrete Design

arXiv:2610.08367v1 Announce Type: new Abstract: Several discrete design tasks, such as molecular discovery, require diverse collections of useful candidates at low computational cost. High validity a...

By Marcus Vukojevic, Erik Nielsen, Veronica Lachi, Andrea Passerini, Giovanni Iacca

benchmarks

arXiv Machine Learning

Oct 7

An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations

arXiv:2610.07211v1 Announce Type: cross Abstract: Systems of equations arise in a wide range of scientific and engineering applications. The present work focuses on solvers for general systems of equ...

By Yuhuang Meng, Jing Zhao, Alexander Heinlein

benchmarks safety

arXiv Machine Learning

Oct 7

Evaluating Behavioral Context for Interpretable IAM Policy Risk Scoring in Cloud Environments

arXiv:2610.07345v1 Announce Type: cross Abstract: IAM policy analysis typically emphasizes the authorization capabilities encoded in a policy, but security analyst review priority may also depend on...

By Yassin Elsharkawy

benchmarks

arXiv Machine Learning

Oct 7

Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds

arXiv:2610.07417v1 Announce Type: cross Abstract: Bayesian Optimization (BO) has become an established methodology for minimizing black-box functions of a vector input. Often, however, this parameter...

By Davide Sartor, Meghan E. Huber, Donghyun Kim, Nathan Wycoff

benchmarks

arXiv Machine Learning

Oct 7

Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings

arXiv:2610.07572v1 Announce Type: cross Abstract: In-context learning (ICL) adapts frozen large multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every q...

By Xi Ding, Naichen Shi, Jiawei Zhang

llms rag multimodal benchmarks

arXiv Machine Learning

Oct 7

On Color Alignment in VAE Latent Spaces and Its Applications

arXiv:2610.07072v1 Announce Type: cross Abstract: Variational autoencoders (VAEs) are a key part of modern text-to-image models, which generate images within their latent space. VAEs are known to dis...

By Julian D. Santamaria, Kai Wang, Jes\'us Malo, Javier Vazquez-Corral, Alexandra G\'omez-Villa

diffusion benchmarks safety

arXiv Machine Learning

Oct 7

RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing

arXiv:2610.07720v1 Announce Type: cross Abstract: Multi-reference image generation requires preserving the appearance of multiple subjects while composing them into a coherent scene. However, existin...

By Wanning He, Yuyao Zhang, Yu-Wing Tai

llms diffusion fine-tuning benchmarks

arXiv Machine Learning

Oct 7

Scen-Opt: A Scenario Optimization Toolbox for Data-Driven Convex Programming

arXiv:2610.07846v1 Announce Type: cross Abstract: The scenario approach is a well-established statistical framework for data-driven decision-making. In particular, in data-driven optimization, the sc...

By Ben Wooding, Simone Garatti, Marco C. Campi, Abolfazl Lavaei

benchmarks

arXiv Machine Learning

Oct 7

Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective

arXiv:2610.07894v1 Announce Type: cross Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is in...

By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Borui Jiang, Bo Han

llms benchmarks safety

arXiv Machine Learning

Oct 7

UNREAL: Unifying Retrieval and Long-Context with a Single Model

arXiv:2610.08463v1 Announce Type: cross Abstract: Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an...

By Edan Kinderman, Elad Hoffer, Yochai Blau, Brian Chmiel, Ron Banner, Daniel Soudry, Boris Ginsburg

llms rag benchmarks

arXiv Machine Learning

Oct 7

PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning

arXiv:2509.18169v4 Announce Type: replace Abstract: Tasks on complex systems require high-precision numerical computation to support decisions. However, current large language models (LLMs), even wit...

By Jingyuan Fan, Purui Liu, Hengbo Xiao, Yuxuan Zheng, Jingzhao Zhang, Chao Lu, Guannan He

llms agents fine-tuning benchmarks

arXiv Machine Learning

Oct 7

Variational Physics-Informed Ansatz for Reconstructing Hidden Interaction Networks from Steady States

arXiv:2512.13708v2 Announce Type: replace Abstract: Inferring interaction structure from steady-state observations is a central inverse problem when transient trajectories are unavailable. Here we fo...

By Kaiming Luo

benchmarks

arXiv Machine Learning

Oct 7

MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning

arXiv:2605.09364v2 Announce Type: replace Abstract: This paper investigates robust representation learning in offline goal-conditioned reinforcement learning (GCRL). Particularly in sparse reward sce...

By Valliappan Chidambaram Adaikkappan, Sai Rajeswar, Pietro Mazzaglia, David Meger

diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 7

Systematic Evaluation of TabPFN-TS and Chronos-2 for Zero-Shot Heat Load Forecasting in District Heating Networks

arXiv:2608.20024v2 Announce Type: replace Abstract: District heating energy hubs require reliable heat load forecasts for efficient operational scheduling. Forecasting models trained on historical da...

By Ben Spoek, Karim K. Ben Hicham, Kai Derzsi, Philipp Althaus, Alexander Mitsos, Dirk M\"uller

benchmarks

arXiv Machine Learning

Oct 7

SketchSSM: Write to the Full State, Read from a Compact Sketch

arXiv:2609.33051v2 Announce Type: replace Abstract: Hybrid-attention models replace most softmax attention layers with linear attention, reducing KV-cache growth and enabling larger decode batches wh...

By Omin Kwon, JoongWon Shin, Minseo Kim, Kurt Keutzer, Sehoon Kim, Jae W. Lee

efficiency benchmarks

arXiv Machine Learning

Oct 7

Consideration Circuits: Depth Separation and Universality Beyond a Single Softmax

arXiv:2610.04143v2 Announce Type: replace Abstract: Most feature-based choice models, classical and deep, score items and apply a single softmax. We introduce consideration circuits (CC), feature-bas...

By Junjie Xiao, Huiwen Jia

benchmarks

arXiv Machine Learning

Oct 7

To Learn is to Wander: Learning Across Graphs and Tasks with Random Walks

arXiv:2610.06694v2 Announce Type: replace Abstract: Graph foundation models aim to transfer across graphs, feature spaces, relational schemas, and prediction tasks, yet existing approaches typically...

By Louis Tichelman, Xingyue Huang, Jinwoo Kim, \.Ismail \.Ilkan Ceylan

benchmarks

arXiv Machine Learning

Oct 7

Improving Mixup Calibration with Wasserstein Distributionally Robust Optimization

arXiv:2506.17874v3 Announce Type: replace-cross Abstract: In many real-world applications, ensuring the robustness and stability of deep neural networks (DNNs) is crucial, particularly for image clas...

By Jiaming Hu, Yeping Jin, Debarghya Mukherjee, Ioannis Ch. Paschalidis

computer-vision benchmarks

Get the daily briefing