Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,270 stories · RSS feed

arXiv Machine Learning

Jul 7

RES-DARE: Failure-Aware Expert Adaptation and Rollback-Safe Self-Repair for Intrusion Detection

arXiv:2607. 02687v1 Announce Type: cross Abstract: Intrusion detection systems are often trained under static benchmark conditions, although deployed network environments are affected by traffic drift, sensor noise, changing workloads, and evolving attack behaviour.

By Rahil Aftab, Anyash Prasad, Soumya Mazumdar, Vineet Kumar Rakesh, Tapas Samanta

benchmarks safety

arXiv Machine Learning

Jul 7

Provable Pruning for Efficient 3D Gaussian Splatting via Coresets

arXiv:2607. 02721v1 Announce Type: cross Abstract: 3D Gaussian Splatting (3DGS) enables high-quality real-time novel-view synthesis, but practical scenes often contain millions of Gaussians, making compression essential for deployment on limited hardware.

By Waseem Mousa, Alaa Maalouf

fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 7

EvoOtter: Evolutionary Reproduction Test Generator

arXiv:2607. 02854v1 Announce Type: cross Abstract: Before fixing an issue, it is useful to first reproduce it by generating a bug reproduction test (BRT).

By Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar, Martin Hirzel

llms benchmarks

arXiv Machine Learning

Jul 7

ParEVO: Synthesizing Code for Irregular Data: High-Performance Parallelism through Agentic Evolution

arXiv:2603. 02510v2 Announce Type: replace Abstract: The transition from sequential to parallel computing is essential for modern high-performance applications but is hindered by the steep learning curve of concurrent programming.

By Liu Yang, Zeyu Nie, Andrew Liu, Felix Zou, Deniz Altinb\"uken, Amir Yazdanbakhsh, Quanquan C. Liu

llms agents fine-tuning benchmarks

arXiv Machine Learning

Jul 7

Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models

arXiv:2506. 07468v4 Announce Type: replace Abstract: Conventional large language model (LLM) safety alignment relies on a reactive, disjoint loop: attackers exploit a static model, then defenders patch exposed vulnerabilities.

By Mickel Liu, Liwei Jiang, Yancheng Liang, Simon Shaolei Du, Yejin Choi, Tim Althoff, Natasha Jaques

llms agents reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 7

An Intervention-Based Framework for Shortcut Diagnosis in Spoofing Countermeasures

arXiv:2607. 03150v1 Announce Type: cross Abstract: While deepfake audio detection systems achieve high performance in controlled benchmarks, their reliability often diminishes in the wild.

By Santiago Rubio, Pilar Bello, Dayana Ribas, Antonio Miguel, Eduardo Lleida, Alfonso Ortega

benchmarks

arXiv Machine Learning

Jul 7

GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Text

arXiv:2312. 15320v3 Announce Type: replace-cross Abstract: Individuals with suspected rare genetic disorders often undergo multiple clinical evaluations, imaging studies, laboratory tests, and genetic tests over a prolonged period of time, a process commonly described as the diagnostic odyssey.

By Da Wu, Zhanliang Wang, Hongzhuo Chen, Jingye Yang, Cong Liu, Tzung-Chien Hsieh, Elaine Marchi, Justin Blair, Peter Krawitz, Chunhua Weng, Wendy Chung, Gholson J. Lyon, Ian D. Krantz, Jennifer M. Kalish, Kai Wang

llms multimodal benchmarks

arXiv Machine Learning

Jul 7

Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

arXiv:2607. 04647v1 Announce Type: cross Abstract: Scalable Bayesian inference for generalized linear mixed models (GLMMs) provides uncertainty-aware analysis of correlated longitudinal data, but existing scalable approaches largely assume low-dimensional tabular predictors and do not directly accommodate high-dimensional modalities such as images and text.

By Yuankang Zhao, Youngsoo Baek, Felipe A. Medeiros, Samuel Berchuck, Matthew M. Engelhard

multimodal benchmarks

arXiv AI

Jul 7

Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantification

arXiv:2607. 05393v1 Announce Type: cross Abstract: Time-domain surveys generate many transient candidates, making Real-Bogus classification a critical step in automated discovery pipelines.

By Rapha\"el Bonnet-Guerrini, Bruno Sanchez, Dominique Fouchez, Benjamin Racine, Maya Guy, Mariam Sabalbal, Manal Yassine, Vincenzo Piuri

benchmarks

arXiv Machine Learning

Jul 7

MTEB-PT: A Text Embedding Benchmark for Brazilian Portuguese

arXiv:2607. 04581v1 Announce Type: cross Abstract: Text embeddings for Portuguese have no dedicated benchmark: evaluation rests on translated corpora such as English MS MARCO or on thin multilingual coverage, with native tasks scattered and unconsolidated.

By Tardelli Ronan Coelho Stekel

rag benchmarks

arXiv Machine Learning

Jul 7

FLASH: Flexible Learning of Adaptive Sampling from History in Temporal Graph Neural Networks

arXiv:2504. 07337v2 Announce Type: replace Abstract: Aggregating temporal signals from historic interactions is a key step in future link prediction on dynamic graphs.

By Or Feldman, Krishna Sri Ipsit Mantri, Carola-Bibiane Sch\"onlieb, Chaim Baskin, Moshe Eliasof

benchmarks

arXiv Machine Learning

Jul 7

Exploring Convolutional Neural Processes for Weather Downscaling

arXiv:2607. 04190v1 Announce Type: new Abstract: Global reanalysis products such as ERA5-Land provide spatially complete weather fields but at resolutions too coarse for local applications, particularly in mountainous regions where temperature can vary by several degrees over short distances.

By Francisco Passos

benchmarks

arXiv Machine Learning

Jul 7

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

arXiv:2607. 04189v1 Announce Type: new Abstract: Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence.

By Liyang Yuan, Yibo Yang, Dandan Guo, Peter Richtarik, Zhouchen Lin

benchmarks safety

arXiv Machine Learning

Jul 7

MARLIN: De Novo Molecular Structure Elucidation from Tandem Mass Spectra without a Ground-Truth Formula

arXiv:2607. 04774v1 Announce Type: new Abstract: Untargeted tandem mass spectrometry (MS/MS) detects thousands of small molecules per biological sample, yet most go unidentified because they are absent from spectral libraries.

By Xujun Che, Xiuxia Du, Depeng Xu

llms diffusion benchmarks

arXiv Machine Learning

Jul 7

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment

arXiv:2607. 01983v1 Announce Type: cross Abstract: Robust 3D object detection under adverse weather remains a critical hurdle for autonomous driving.

By Shuyao Li, Chuanxing Geng, Heyang Sun, Qiang Zhou, Jingjing Gu

agents diffusion computer-vision robotics benchmarks safety

arXiv Machine Learning

Jul 7

The Bayesian Approach to Continual Learning: An Overview

arXiv:2507. 08922v3 Announce Type: replace-cross Abstract: Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps.

By Tameem Adel

fine-tuning benchmarks

arXiv Machine Learning

Jul 7

CIPHER: Causal Intervention Pathways for Healthcare Equity and Robustness

arXiv:2607. 02596v1 Announce Type: cross Abstract: Deep learning models for medical diagnosis frequently exhibit substantial performance disparities across sensitive subgroups (e.

By Xinyu Jia, Weidong Guo, Wangyuan Zhao, Yi Guo, Zeju Li, Yuanyuan Wang

diffusion benchmarks

arXiv Machine Learning

Jul 7

Schedulable Job-Level Dependencies for Cause-Effect Chains via Graph Neural Networks

arXiv:2607. 02624v1 Announce Type: cross Abstract: Modern automotive software architectures comprise large sets of mixed-criticality functions executing on shared multi-core platforms with strict real-time and end-to-end timing requirements.

By Silviu S. Craciunas, Christian Hakert, Jian-Jia Chen, Zden\v{e}k Hanz\'alek, Paul Pop

benchmarks

arXiv AI

Jul 7

SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing

arXiv:2607. 04616v1 Announce Type: cross Abstract: Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes.

By Stone Tao, Jie Xu, Hesam Rabeti, Yashraj Narang, Yijie Guo, Iretiayo Akinola

reinforcement-learning robotics benchmarks

arXiv AI

Jul 7

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

arXiv:2607. 03644v1 Announce Type: cross Abstract: Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, radar, gravity, and elemental composition.

By Ayush Prasad, Swarnalee Mazumder

llms computer-vision multimodal benchmarks

Get the daily briefing