Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,504 stories · RSS feed

arXiv Machine Learning

Jun 25

A 3D-Printable Dataset for Fair Testing and Comparisons of Tactile Sensors

arXiv:2606. 25886v1 Announce Type: cross Abstract: Existing texture datasets for tactile sensing primarily consist of sensor readings from a specific sensor interacting with available surfaces/objects rather than describing the textures themselves, limiting fair comparison between tactile sensors and hindering reproducible research.

By Dexter R. Shepherd, Nicolas Herzig, Phil Husbands, Andrew Philippides, Chris Johnson, William Kimbell

benchmarks

arXiv Machine Learning

Jun 25

Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models

arXiv:2606. 24898v1 Announce Type: new Abstract: Looped language models turn hidden states into runtime state: each state is decoded for prediction and fed back into future computation.

By Rituraj Sharma, Tu Vu

llms benchmarks

arXiv Machine Learning

Jun 25

TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

arXiv:2606. 24039v1 Announce Type: cross Abstract: Robotics increasingly relies on GPUs for parallel simulation, large-scale learning, and neural-network inference.

By Gabriel Bravo-Palacios, Jianghan Zhang, Zachary Pestrikov, Brian Plancher, Thomas Lew

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 25

On-Device Neural Architecture Search

arXiv:2606. 24900v1 Announce Type: new Abstract: This paper proposes a new approach to near-sensor computing, in which a lightweight Neural Architecture Search (NAS) is performed directly on the deployment device to find the best tiny neural architecture for analyzing the real-time data acquired through sensors.

By Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo, Antonio Frisoli, Claudio Loconsole

benchmarks

arXiv Machine Learning

Jun 25

Stable-Shift: Biologically Structured Prediction of Transcriptional Responses to Unseen Gene Perturbations

arXiv:2606. 24940v1 Announce Type: cross Abstract: Predicting transcriptional responses to genetic perturbations could reduce the experimental burden of functional genomics, but extrapolation to genes that were never perturbed during training remains difficult.

By Sajib Acharjee Dip, Liqing Zhang

benchmarks

arXiv Machine Learning

Jun 25

Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

arXiv:2606. 25449v1 Announce Type: cross Abstract: A language model's memory can be worse than having no memory at all.

By Alex Kwon

llms benchmarks

arXiv Machine Learning

Jun 25

Frequency Domain Reservoir Computing

arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.

By Klaus Schertler, Xiomara Runge, Andrea Ceni, David Kappel, Claudio Gallicchio

llms rag benchmarks

arXiv Machine Learning

Jun 25

KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Tagging

arXiv:2512. 07420v3 Announce Type: replace-cross Abstract: Jet identification plays a central role in analyzing data from high-energy collider experiments.

By Md Raqibul Islam, Adrita Khan, Mir Sazzat Hossain, Choudhury Ben Yamin Siddiqui, Md. Zakir Hossan, Tanjib Khan, M. Arshad Momen, Amin Ahsan Ali, AKM Mahbubur Rahman

benchmarks safety

arXiv Machine Learning

Jun 25

AeroCast: Probabilistic 3D Trajectory Prediction for Non-Cooperative Aerial Obstacles via Transformer-MDN Architecture

arXiv:2606. 25122v1 Announce Type: cross Abstract: Autonomous aerial vehicles operating in shared airspace must predict the future positions of non-cooperative obstacles to plan evasive maneuvers before a collision becomes unavoidable.

By Syed Izzat Ullah, Jose Baca

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 25

Benchmarking the Alignment of Data-Quality Metrics, Human Judgment and Land-Cover Segmentation Performance for Earth Observation

arXiv:2606. 25128v1 Announce Type: cross Abstract: Volume and quality of datasets are crucial for deep learning model training, yet they are often constrained by availability and data acquisition costs.

By \"Umit Mert \c{C}a\u{g}lar, Alptekin Temizel

computer-vision benchmarks safety

arXiv Machine Learning

Jun 25

LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search

arXiv:2606. 25039v1 Announce Type: new Abstract: Recovering governing Ordinary Differential Equations (ODEs) from data is a central challenge in modeling dynamical systems across scientific domains.

By Nikhil Abhyankar, Sha Li, Sanchit Kabra, Naren Ramakrishnan, Yulia Gel, Chandan K. Reddy

llms benchmarks

arXiv Machine Learning

Jun 25

When Multi-Sensor Fusion Fails to Generalize: Cattle Posture Classification Under Animal-Level and Temporal Distribution Shift

arXiv:2606. 24986v1 Announce Type: new Abstract: Automated cattle posture-classification systems frequently report near-perfect accuracy, yet their robustness under realistic deployment conditions remains largely unknown.

By Leutrim Uka, Severino Pinto, Gundula Hoffmann, Marina M. -C. H\"ohne

multimodal benchmarks

arXiv Machine Learning

Jun 25

Improving Factuality of 3D Brain MRI Report Generation with Paired Image-domain Retrieval and Text-domain Augmentation

arXiv:2411. 15490v2 Announce Type: replace-cross Abstract: Acute ischemic stroke (AIS) requires time-critical decision-making, where inaccurate interpretation of neuroimaging findings can lead to irreversible disability.

By Junhyeok Lee, Yujin Oh, Dahyoun Lee, Hyon Keun Joh, Chul-Ho Sohn, Sung Hyun Baik, Cheol Kyu Jung, Jung Hyun Park, Kyu Sung Choi, Byung-Hoon Kim, Jong Chul Ye

llms rag diffusion multimodal benchmarks safety

arXiv Machine Learning

Jun 25

Towards Robust EEG Decoding Based on Riemannian Self-Attention

arXiv:2606. 25456v1 Announce Type: new Abstract: Brain-Computer Interface (BCI) based on electroencephalography (EEG) enables direct interaction between the brain and external environments and has significant applications in assistive technologies, medical rehabilitation, and entertainment.

By Shaocheng Jin, Tao Zhou, Rui Wang, Ziheng Chen, Xiaoqing Luo, Xiaojun Wu, Josef Kittler

benchmarks

arXiv Machine Learning

Jun 25

LLM Program Optimization via Retrieval Augmented Search

arXiv:2501. 18916v2 Announce Type: replace Abstract: Recent work has demonstrated the potential of large language models (LLMs) for program optimization, a key challenge in programming languages.

By Sagnik Anupam, Alexander Shypula, Osbert Bastani

llms rag benchmarks safety

arXiv Machine Learning

Jun 25

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

arXiv:2605. 23434v2 Announce Type: replace Abstract: Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs).

By Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

diffusion benchmarks

arXiv Machine Learning

Jun 25

Leaking Circuit Secrets: Gradient Leakage Attacks on Graph Neural Networks

arXiv:2606. 25589v1 Announce Type: new Abstract: As graph neural networks (GNNs) become standard tools for critical tasks in circuit design and analysis, their security and privacy risks require careful attention.

By Rupesh Raj Karn, Johann Knechtel, Ozgur Sinanoglu

efficiency benchmarks safety

arXiv Machine Learning

Jun 25

Learning with a Single Rollout via Monte Carlo Pass@k Critic

arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.

By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Certified Robust Invariant Polytope Training in Neural Controlled ODEs

arXiv:2408. 01273v3 Announce Type: replace Abstract: We propose a framework for training neural network controllers with certified robust forward invariant polytopes.

By Akash Harapanahalli, Samuel Coogan

rag benchmarks

arXiv Machine Learning

Jun 25

DFMU: Data-Frugal Machine Unlearning

arXiv:2606. 25410v1 Announce Type: new Abstract: Machine unlearning is an emerging domain that ensures the safe removal of elements (includes concepts, attributes, entity and class) from the trained model along with least drop in model performance.

By Sajith U, Prateek Keserwani

efficiency benchmarks

Get the daily briefing