Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,633 stories · RSS feed

arXiv AI

Jul 21

MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models

arXiv:2607. 18006v1 Announce Type: cross Abstract: Large language models achieve strong reasoning performance, but often at prohibitive training cost - a challenge that is especially acute for compact models ($\leq 4 \, \mathrm{B}$ parameters) trained under limited budgets.

By Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Zifeng Ding, Volker Tresp, Yunpu Ma

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 21

Lookahead Branching for Neural Network Verification

arXiv:2607. 17290v1 Announce Type: cross Abstract: In this work, we investigate the effect of lookahead branching strategies in neural network verification.

By Liam Davis, Duo Zhou, Huan Zhang, Guy Katz, Clark Barrett, Haoze Wu

benchmarks

arXiv AI

Jul 21

Fourier Geometric Wind Power Forecasting with Numerical Weather Prediction

arXiv:2607. 17095v1 Announce Type: new Abstract: Accurate short-term wind power forecasting is essential for grid stability and operational planning, yet remains challenging due to the complex interactions between atmospheric conditions and turbine dynamics.

By Shiyuan Piao, Fan Zehui, Yang Liu, Hong Cheng, Juepeng Zheng, Jie Zhou, Fugee Tsung

multimodal benchmarks

arXiv AI

Jul 21

Short-Term Electricity Demand Forecasting for New England: A Comprehensive Machine Learning Benchmark with Weather, Calendar, and COVID-19 Indicators

arXiv:2606. 20918v2 Announce Type: replace-cross Abstract: Accurate short-term electricity demand forecasting is critical for reliable power system operation, energy market planning, and infrastructure optimization.

By Reza Ghanavati, Behrooz Mosallaei

llms benchmarks

arXiv Machine Learning

Jul 21

Bridging battery design and health assessment through virtual sensing and physics-informed learning

arXiv:2607. 16864v1 Announce Type: new Abstract: Supercharging of lithium-ion batteries (LiBs) requires robust health monitoring to ensure durability, safety, and user confidence, particularly for emerging vehicle-to-grid applications with bidirectional energy flows.

By Wendi Guo, S{\o}ren Byg Vilsen, Daniel Ioan Stroe, Yaqi Li, Yicun Huang, Ashima Verma, Daniel Brandell

rag diffusion benchmarks safety

arXiv Machine Learning

Jul 21

Interpretable Anomaly and Drift Detection with Gaussian Mixture Models

arXiv:2607. 16811v1 Announce Type: new Abstract: We revisit Gaussian Mixture Models (GMMs) as a lightweight, interpretable tool for anomaly detection and, in particular, for detecting distributional drift in data streams.

By Behnam Asadi

benchmarks

arXiv Machine Learning

Jul 21

HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling

arXiv:2607. 16882v1 Announce Type: new Abstract: Time series forecasting (TSF) is vital to many applications, yet existing models often struggle to capture the heterogeneous long-range global patterns and short-range local variations in multivariate time series.

By Wenqiang Ma, Chen Cheng, Xue Cheng, Jiarui Ye

llms benchmarks

arXiv AI

Jul 21

Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning

arXiv:2607. 16534v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment.

By Vincent Taboga, Justin Veilleux, Doseok Jang, Anushree Rankawat, Pierre-Luc Bacon

reinforcement-learning benchmarks

arXiv AI

Jul 21

Interactive Task Alignment as a POMDP

arXiv:2607. 16412v1 Announce Type: new Abstract: Current benchmarks for language models primarily evaluate execution on fully specified tasks.

By Andy Dai, Zexue He, Zhenyu Zhang, Alex Pentland, Jiaxin Pei

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 21

Spatiotemporal Facial Action Unit Detection using Twin Cycle Autoencoders for Driver Monitoring

arXiv:2607. 16760v1 Announce Type: cross Abstract: Driver monitoring systems (DMS) increasingly rely on facial cues to infer drowsiness, distraction, and cognitive load in real time.

By Sai Sidharth D

efficiency benchmarks safety

arXiv AI

Jul 21

Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory

arXiv:2607. 17545v1 Announce Type: new Abstract: Language agents depend on memory across interactions.

By Qingcan Kang, Mingyang Liu, Shixiong Kai, Kaichao Liang, Zhentao Tang, Yuqi Cui, Tao Zhong, Mingxuan Yuan

llms agents benchmarks safety

arXiv Machine Learning

Jul 21

Decoder-Preserving Sparse Autoencoders: Which Readouts Survive Sparse Compression?

arXiv:2607. 17425v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) compress model activations into sparse codes, but equal reconstruction error and sparsity can preserve different linearly decodable signals.

By Aniket Deshpande

llms efficiency benchmarks

arXiv AI

Jul 21

LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Models

arXiv:2607. 16305v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have achieved strong progress in multimodal understanding.

By Zeyu Xu, Xingzhong Hou, Pengkai Guo, Siling Lin, Xiao Xu, Menghua Zhai, Haoyu Chen, Yunke Zhang, Fei Huang

llms rag multimodal benchmarks

arXiv AI

Jul 21

"Not in My Backyard": LLMs Uncover Online and Offline Social Biases Against Homelessness

arXiv:2508. 13187v4 Announce Type: replace-cross Abstract: Homelessness is a persistent social challenge, impacting millions worldwide.

By Jonathan A. Karr Jr., Benjamin F. Herbst, Matthew L. Sisk, Xueyun Li, Ting Hua, Matthew Hauenstein, Georgina Curto, Nitesh V. Chawla

llms benchmarks safety

arXiv AI

Jul 21

OFD-Net: Teacher-Free Reliable Semi-supervised Medical Image Segmentation with Orthogonal Feature Disentanglement Net of Foreground-Background

arXiv:2607. 16705v1 Announce Type: cross Abstract: Semi-supervised learning (SSL) is an effective solution for medical image segmentation with limited annotations.

By Shao-feng Jiang, Zhe-yang Jing, Qin Lu, Huan-huan Shi, Zhen Chen, Cong-xuan zhang, Chen Yi

computer-vision benchmarks safety

arXiv AI

Jul 21

FST.ai 2.5: Explainable and Uncertainty-Aware AI for Olympic and Para-Taekwondo Decision Support, Athlete Digital Twins, and Federation-Scale Analytics

arXiv:2607. 16597v1 Announce Type: new Abstract: The rapid digitalisation of elite sport has created new opportunities for integrating artificial intelligence (AI), performance analytics, and decision-support systems into athlete development and competition management.

By Keivan Shariatmadar, Ahmad Osman, Ramin Rey

benchmarks

arXiv AI

Jul 21

CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning

arXiv:2607. 17568v1 Announce Type: cross Abstract: Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups.

By Zhiren Gong, Zihao Zeng, Zijie Wang, Tiantong Wang, Chau Yuen, Wei Yang Bryan Lim

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 21

Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA

arXiv:2607. 17834v1 Announce Type: cross Abstract: Endoscopic visual question answering (VQA) increasingly asks complex questions that combine several endoscopic answer components rather than isolated factual queries.

By Yuhao Liu, Cheng Zhao, Guanghui Yue

nlp multimodal benchmarks

arXiv Machine Learning

Jul 21

Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift

arXiv:2607. 17524v1 Announce Type: cross Abstract: We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task.

By Zitong Huang, Gustavo Lucas Carvalho, Deqing Fu, Robin Jia

nlp fine-tuning benchmarks

arXiv AI

Jul 21

Multi-modal cross-domain mixed fusion model with dual disentanglement for fault diagnosis under unseen working conditions

arXiv:2512. 24679v2 Announce Type: replace Abstract: Intelligent fault diagnosis has become an indispensable technique for ensuring machinery reliability.

By Pengcheng Xia, Yixiang Huang, Chengjin Qin, Chengliang Liu

fine-tuning multimodal benchmarks

Get the daily briefing