Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,997 stories · RSS feed

arXiv Machine Learning

Jun 18

Fair Cognitive Impairment Detection Through Unlearning

arXiv:2606. 18571v1 Announce Type: new Abstract: Mild Cognitive Impairment (MCI) is a medical condition characterized by a noticeable decline in memory, language, or thinking abilities.

By William Nguyen, Jiali Cheng, Hadi Amiri

rag multimodal benchmarks

arXiv AI

Jun 18

Hilbert-Geo: Solving Solid Geometric Problems by Neural-Symbolic Reasoning

arXiv:2605. 16385v3 Announce Type: replace-cross Abstract: Geometric problem solving, as a typical multimodal reasoning problem, has attracted much attention and made great progress recently, however most of works focus on plane geometry while usually fail in solid geometry due to 3D spatial diagrams and complex reasoning.

By Ruoran Xu, Haoyu Cheng, Bin Dong, Qiufeng Wang

llms multimodal benchmarks

arXiv AI

Jun 18

WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks

arXiv:2604. 06367v2 Announce Type: replace-cross Abstract: Web agents automate browser tasks, ranging from simple form completion to complex workflows like ordering groceries.

By Guruprasad Viswanathan Ramesh, Asmit Nayak, Basieem Siddique, Kassem Fawaz

llms agents multimodal benchmarks safety

arXiv AI

Jun 18

A Controlled Benchmark of Quantum-Latent GAN Augmentation for Brain MRI

arXiv:2606. 18970v1 Announce Type: cross Abstract: Medical image classification is often constrained by limited labeled data, motivating generative augmentation; recently, quantum generative models have been proposed for this purpose, frequently reporting accuracy gains.

By Syed Mujtaba Haider, Silvia Figini

diffusion computer-vision benchmarks

arXiv AI

Jun 18

CAOA -- Completion-Assisted Object-CAD Alignment

arXiv:2606. 18429v1 Announce Type: cross Abstract: Accurately aligning CAD models to their corresponding objects in indoor RGB-D scans is a central challenge in 3D semantic reconstruction.

By Hiranya Garbha Kumar, Minhas Kamal, Balakrishnan Prabhakaran

computer-vision benchmarks safety

arXiv AI

Jun 18

Code-Augur: Agentic Vulnerability Detection via Specification Inference

arXiv:2606. 18619v1 Announce Type: cross Abstract: The advent of agentic vulnerability detection is already becoming a watershed moment for software security.

By Zhengxiong Luo, Mehtab Zafar, Dylan Wolff, Abhik Roychoudhury

llms agents benchmarks

arXiv AI

Jun 18

Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech

arXiv:2603. 15988v3 Announce Type: replace-cross Abstract: Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies.

By Jaesung Bae, Xiuwen Zheng, Minje Kim, Chang D. Yoo, Mark Hasegawa-Johnson

fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 18

Quantum Annealing Enhanced Reinforcement Learning for Accurate Remaining Useful Lifetime Prediction

arXiv:2606. 18503v1 Announce Type: new Abstract: Remaining useful life (RUL) estimation is central to predictive maintenance, where an unplanned failure can cost far more than the asset itself.

By Manoranjan Gandhudi, Arunkumar V., G. R. Anil, Gangadharan G. R

rag reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

SCAN: Enhance Time Series Anomaly Detection via Multi-Scale Neighborhood-Centered Clustering

arXiv:2606. 19255v1 Announce Type: new Abstract: Time series anomaly detection plays a crucial role in a wide range of real-world applications.

By Xingze Zheng, Hanyin Cheng, Siyuan Wang, Yiting Hao, Peng Chen, Yuan Jun, Yang Shu

benchmarks

arXiv AI

Jun 18

Simulating Hate Speech Cascades with Multi-LLM Agents: Empirical Grounding, Modeling Fidelity, and Intervention Strategies

arXiv:2606. 18264v1 Announce Type: cross Abstract: Faithful modeling of hateful content propagation on online platforms remains an open problem for moderation research.

By Fan Huang

llms agents diffusion benchmarks safety

arXiv AI

Jun 18

LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection

arXiv:2605. 17986v3 Announce Type: replace-cross Abstract: AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools.

By Lei Zhao, Abhay Bhaskar, Edgar Dobriban

llms agents benchmarks

arXiv AI

Jun 18

SAERec: Constructing Fine-grained Interpretable Intents Priors via Sparse Autoencoders for Recommendation

arXiv:2606. 18897v1 Announce Type: cross Abstract: Intent-based recommender systems have gained significant attention for improving accuracy and interpretability by modeling the underlying motivations behind user behaviors.

By Jiangnan Xia, Xuansheng Wu, Yu Yang, Xin Wang, Ninghao Liu

llms rag diffusion benchmarks safety

arXiv Machine Learning

Jun 18

Context-Aware Optimization of Follow-Up Intervals for Type 2 Diabetes Care Using Markov Decision Processes

arXiv:2606. 19092v1 Announce Type: cross Abstract: Chronic disease management relies on regular patient-provider interactions to follow-up on disease progression and control.

By Parisa Lotfibagha, Kristen Miller, William J. Gallagher, Elizabeth B. Selden, Muge Capan

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

Spatiotemporal downscaling and nowcasting of urban land surface temperatures with deep neural networks

arXiv:2605. 13566v2 Announce Type: replace Abstract: Land Surface Temperature (LST) is a key variable for various applications, such as urban climate and ecology studies.

By Solomiia Kurchaba, Angela Meyer

benchmarks safety

arXiv AI

Jun 18

NAVI-Orbital: First In-Orbit Demonstration of a Zero-Shot Vision-Language Model for Autonomous Earth Observation

arXiv:2606. 18271v1 Announce Type: new Abstract: As Earth Observation data generation outpaces downlink bandwidth and human-in-the-loop processing, a widening gap has emerged between onboard collection and actionable ground intelligence.

By Juan Manuel Delfa Victoria, Taran Cyriac John, Andrew W. Herson

llms agents fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 18

Pointwise is Pointless? A Multimodal Ablation Study for Precipitation Nowcasting with Graph Neural Networks

arXiv:2606. 18436v1 Announce Type: cross Abstract: Sparse point observations are increasingly available for precipitation nowcasting, but it is unclear how much they improve dense radar-field forecasts.

By Oph\'elia Miralles, M\'at\'e Mile, Christoffer Artturi, Thomas Nipen, Ivar Seierstad

multimodal benchmarks

arXiv Machine Learning

Jun 18

Investigation of Neural Network Methods for Reconstruction and Classification of Texture Images Under Conditions of Incomplete Information

arXiv:2204. 14224v3 Announce Type: replace-cross Abstract: The automated analysis of heterogeneous natural textures is frequently hindered by physical damage and data loss, presenting a significant challenge to computer vision.

By Galymzhan Abdimanap, Kairat Bostanbekov, Abdelrahman Abdallah, Anel Alimova, Darkhan Kurmangaliyev, Daniyar Nurseitov, Tatyana Dedova, Larissa Balakay, Serik Nurakynov

llms agents diffusion computer-vision benchmarks safety

arXiv AI

Jun 18

TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering

arXiv:2604. 28076v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced Table Question Answering, where most queries can be answered by extracting information or simple aggregation.

By An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia Ye

llms agents nlp benchmarks

arXiv Machine Learning

Jun 18

Global Offshore Wind Infrastructure: Deployment and Operational Dynamics from Dense Sentinel-1 Time Series

arXiv:2604. 20822v2 Announce Type: replace-cross Abstract: The offshore wind energy sector is expanding rapidly, increasing the need for independent, high-temporal-resolution monitoring of infrastructure deployment and operation at global scale.

By Thorsten Hoeser, Felix Bachofer, Claudia Kuenzer

computer-vision benchmarks

arXiv Machine Learning

Jun 18

Measurement noise limits the advantage of nonlinear models over linear models in biomedical prediction

arXiv:2606. 18420v1 Announce Type: new Abstract: On biomedical tabular data, flexible models such as deep networks, gradient-boosted trees, and kernel methods are repeatedly matched or beaten by linear and logistic regression given the same features.

By Marc-Andre Schulz, Kerstin Ritter

benchmarks

Get the daily briefing