Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

17,750 stories · RSS feed

arXiv AI

Jul 7

CuBAS: Information Geometric Curvature-Based Adaptive Sampling for Supervised Classification

arXiv:2607. 03145v1 Announce Type: cross Abstract: The informativeness of a training set is as consequential as its size, yet most sampling strategies remain agnostic to the intrinsic geometry of the data distribution.

By Alexandre L. M. Levada

benchmarks

arXiv Machine Learning

Jul 7

Seeing Through WiFi: Lightweight Human Pose Estimation with Dynamic Kernel Attention

arXiv:2607. 03196v1 Announce Type: cross Abstract: WiFi-based human pose estimation (HPE) enables the detection and interpretation of human body positions and movements without the need for wearable devices while preserving individual privacy concerns.

By Toan D. Gian, Van-Dinh Nguyen, Vo Phi Son, Nhan Thanh Nguyen, Dinh Thai Hoang, Diep N. Nguyen, Nguyen Cong Luong, Symeon Chatzinotas

computer-vision benchmarks safety

arXiv AI

Jul 7

RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities

arXiv:2607. 04729v1 Announce Type: cross Abstract: LLM agents are increasingly applied to vulnerability analysis, but existing benchmarks have not kept pace.

By Tarek Elsayed, Shiping Yang, Eunsong Koh, Sanika Goyal, Vincent Huang, Paul Ngo, Nathan Young, Mohammad Omidvar Tehrani, Alvyn Kang, Arnell Kang, Zeyu Chen, Ang\'elica Moreira, Xuan Feng, Angel X. Chang, Nick Sumner, Steven Y. Ko

llms agents benchmarks safety

arXiv AI

Jul 7

Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs

arXiv:2607. 04371v1 Announce Type: new Abstract: We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment.

By Akhiad Bercovich, Talor Abramovich, Daniel Afrimi, Shay Aharon, Nir Ailon, Vladimir Anisimov, Omer Ullman Argov, Maor Ashkenazi, Tomer Asida, Nave Assaf, Tomer Bar Natan, Alexander Bukharin, Grzegorz Chlebus, Marcin Chochowski, Eric Chung, Mohammad Dabbah, Carlo del Mundo, Ewa Dobrowolska, Ido Galil, Yaniv Galron, Amnon Geifman, Yonatan Geifman, Izik Golan, Alex Gronskiy, Tomasz Grzegorzek, Netanel Haber, Lior Kadoch, Grzegorz Karch, Tomer Keren, Abhinav Khattar, Amir Klein, Tugrul Konuk, Roi Koren, Daniel Korzekwa, Shaun Kotek, Konstantinos Krommydas, Itay Levy, Ofri Masad, Yoav Miron, Pavlo Molchanov, Shahar Mor, Zach Moshe, Saurav Muralidharan, Najeeb Nabwani, Besmira Nushi, Mostofa Patwary, Omri Puny, Johannes Rausch, Tomer Ronen, Sepehr Sameni, Itamar Schen, Elad Segal, Daniel Serebrenik, Ido Shahaf, Soumye Singhal, Daniil Sorokin, Sharath Turuvekere Sreenivas, Marta Stepniewska-Dziubinska, Ali Taghibakhshi, Nima Tajbakhsh, Oren Tropp, Dor Tzur, Anna Warno, Yi-Fu Wu, Michal Zawalski, Jiaqi Zeng, Yian Zhang, Ran Zilberstein, Amit Zuker, Ran El-Yaniv

llms agents reinforcement-learning efficiency benchmarks

arXiv AI

Jul 7

DrugAgent: Reliable Multi-Agent Integration of Conflicting Biomedical Evidence for Drug-Target Interaction Assessment

arXiv:2408. 13378v5 Announce Type: replace Abstract: Workflows in drug-target interaction (DTI) assessment require integrating heterogeneous data from predictive models, curated resources, and observations from experimental literature.

By Yoshitaka Inoue, Tianci Song, Xinling Wang, Rui Kuang, Tianfan Fu, Augustin Luna

llms rag agents nlp benchmarks

arXiv Machine Learning

Jul 7

QiVC-Net: Quantum-Inspired Variational Convolutional Network, with Application to Biosignal Classification

arXiv:2511. 05730v2 Announce Type: replace Abstract: In this paper, a learning framework is introduced which incorporates principles of probabilistic inference, variational optimization, and geometry-preserving operations inspired by quantum transformations.

By Amin Golnari, Jamileh Yousefi, Reza Moheimani, Saeid Sanei

benchmarks

arXiv AI

Jul 7

Correlation-Weighted Multi-Reward Optimization for Compositional Generation

arXiv:2603. 18528v2 Announce Type: replace Abstract: Text-to-image models produce images that align well with natural language prompts, but compositional generation has long been a central challenge.

By Jungmyung Wi, Hyunsoo Kim, Donghyun Kim

diffusion benchmarks

arXiv AI

Jul 7

PLACEMEM: Toward a Compute-Aware Memory Plane for Lifelong Agents

arXiv:2607. 04089v1 Announce Type: new Abstract: Lifelong agents need more than larger context windows and better retrieval.

By Sukanta Ganguly

agents benchmarks

arXiv Machine Learning

Jul 7

Geometry-Aware Bayesian Quantification via Compositional Data Analysis

arXiv:2607. 04977v1 Announce Type: new Abstract: Accurately estimating the unknown target label distribution is the critical first step for adapting to label shift.

By Alejandro Moreo, Pablo Gonz\'alez, Juan Jos\'e del Coz

benchmarks

arXiv AI

Jul 7

GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks

arXiv:2607. 05369v1 Announce Type: cross Abstract: For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies?

By Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue, Jonathan Francis, Christian Juette, Peter Schaldenbrand, Muhammet Yunus Seker, Ruwan Wickramarachchi, Uksang Yoo, Guanzhi Wang, Adithyavairavan Murali, Balakumar Sundaralingam, S. Shankar Sastry, Spencer Huang, Yuke Zhu, Linxi "Jim" Fan, Ken Goldberg

agents robotics efficiency benchmarks

arXiv Machine Learning

Jul 7

Sensitivity Sampling with Predictions for k-Means Clustering

arXiv:2607. 04949v1 Announce Type: new Abstract: We study the problem of k-means clustering on large datasets.

By Cristian Boldrin, Fabio Vandin

benchmarks

arXiv Machine Learning

Jul 7

Advances in Neural Controlled Differential Equations

arXiv:2607. 05280v1 Announce Type: new Abstract: Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences.

By Benjamin Walker

benchmarks

arXiv Machine Learning

Jul 7

Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms

arXiv:2607. 02681v1 Announce Type: cross Abstract: Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and federated learning, but contamination and heterogeneity make robust borrowing challenging.

By Ye Tian, Mengchu Li, Marco Avella Medina

benchmarks

arXiv Machine Learning

Jul 7

SleepBand: Single-Source Domain Generalization for Sleep Staging via Physiologically Structured Spectral Modeling

arXiv:2607. 04851v1 Announce Type: cross Abstract: Generalizing sleep staging models to unseen datasets is challenging, and typical domain generalization (DG) methods often rely on multiple source domains or domain labels that are rarely available in practice.

By Zhi Lu, Yang Hu, Yan Chen

benchmarks

arXiv AI

Jul 7

InverseScope: Scalable Activation Inversion for Interpreting Large Language Models

arXiv:2506. 07406v3 Announce Type: replace-cross Abstract: Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research.

By Yifan Luo, Zhennan Zhou, Bin Dong

llms efficiency benchmarks safety

arXiv Machine Learning

Jul 7

Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX

arXiv:2510. 01764v3 Announce Type: replace Abstract: Reinforcement learning (RL) research requires diverse, challenging environments that are both tractable and scalable.

By Waris Radji, Thomas Michel, Hector Piteau

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 7

Training Verifiably Robust Agents Using Set-Based Reinforcement Learning

arXiv:2408. 09112v2 Announce Type: replace Abstract: Reinforcement learning policies parametrized by deep neural networks have achieved strong performance for continuous control, yet even small input perturbations may lead to unpredictable behavior.

By Manuel Wendl, Lukas Koller, Tobias Ladner, Matthias Althoff

agents reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 7

Fortifying Fully Convolutional Generative Adversarial Networks for Image Super-Resolution Using Divergence Measures

arXiv:2404. 06294v2 Announce Type: replace-cross Abstract: Super-Resolution (SR) is a time-hallowed image processing problem that aims to improve the quality of a Low-Resolution (LR) sample up to the standard of its High-Resolution (HR) counterpart.

By Arkaprabha Basu, Kushal Bose, Sankha Subhra Mullick, Anish Chakrabarty, Swagatam Das

diffusion benchmarks safety

arXiv AI

Jul 7

VISOR: Visual Input-based Steering for Output Redirection in Vision-Language Models

arXiv:2508. 08521v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) are increasingly being used in a broad range of applications, bringing their security and behavioral control to the forefront.

By Mansi Phute, Ravikumar Balakrishnan

llms robotics multimodal benchmarks safety

arXiv AI

Jul 7

Query2Diagram: Answering Developer Queries with UML Diagrams

arXiv:2604. 23816v2 Announce Type: replace-cross Abstract: Software documentation frequently becomes outdated or fails to exist entirely, yet developers need focused views of their codebase to understand complex systems.

By Oleg Baryshnikov, Anton M. Alekseev, Sergey I. Nikolenko

llms fine-tuning benchmarks

Get the daily briefing