Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv Machine Learning
Jul 14

Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator

arXiv:2607. 10960v1 Announce Type: new Abstract: Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs), but historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment.

By Wen-Ting Wang
arXiv AI
Jul 14

LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning

arXiv:2607. 10139v1 Announce Type: cross Abstract: Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution.

By Ning Liu
arXiv Machine Learning
Jul 14

Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces

arXiv:2607. 07032v2 Announce Type: replace Abstract: Spectral positional encodings (PEs) for \emph{directed} graphs face two obstacles: magnetic Laplacians require an $O(n^3)$ Hermitian eigendecomposition per potential, and their complex eigenvectors are defined only up to unitary gauge, which prior work handles with basis-invariant architectures.

By Jiaqing Xie, Yuxin Wang
arXiv AI
Jul 14

VehAnchor: Metadata-Free Metric Scale Recovery from Vehicle Cues in Aerial Imagery

arXiv:2603. 04277v2 Announce Type: replace-cross Abstract: Autonomous aerial robots operating in GPS-denied or communication-degraded environments frequently lose access to camera metadata and telemetry, leaving onboard perception systems unable to recover the absolute metric scale of the scene.

By Yifei Chen, Chenqian Le, Jiayi Cheng, Xupeng Chen
arXiv Machine Learning
Jul 14

Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning

arXiv:2607. 11473v1 Announce Type: cross Abstract: In this paper, we propose TECO, a multi-dimensional pruning framework to collaboratively prune the three dimensions (depth, width, and resolution) of convolutional neural networks (CNNs) for better execution efficiency on embedded hardware.

By Hao Kong, Di Liu, Xiangzhong Luo, Shuo Huai, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
arXiv AI
Jul 14

SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

arXiv:2607. 11185v1 Announce Type: new Abstract: Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution.

By Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong
arXiv Machine Learning
Jul 14

Modernizing HEBO: a robust Bayesian optimization baseline for practical heteroskedastic and non-stationary problems

arXiv:2607. 10669v1 Announce Type: new Abstract: Bayesian optimization is increasingly used to guide data-efficient experimentation in chemistry, materials science, and related laboratory settings, but its practical performance depends strongly on how well surrogate-model assumptions match the geometry and noise structure of the underlying objective.

By L. A. Zhukov, E. V. Shaburova, D. V. Antonets