Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,494 stories · RSS feed

arXiv Machine Learning

Jun 3

Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs

arXiv:2606. 02628v1 Announce Type: new Abstract: We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest.

By Aizierjiang Aiersilan

llms efficiency benchmarks safety

arXiv AI

Jun 3

From Long News to Accurate Forecast: Importance-Aware Fusion and PRM-Guided Reflection for Time Series Forecasting

arXiv:2606. 03097v1 Announce Type: new Abstract: Incorporating news into time series forecasting is appealing because news can reveal abrupt exogenous events that historical values alone cannot recover.

By Mingyang Liu, Qingcan Kang, Yuke Wang, Shixiong Kai, Kaichao Liang, Hui-Ling Zhen, Tao Zhong, Mingxuan Yuan, Linqi Song

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 3

Forecasting Conceptual Diffusion in Science: The Case of Quantum Computing

arXiv:2606. 03919v1 Announce Type: cross Abstract: Understanding and anticipating scientific change requires models that distinguish between endogenous consolidation and exogenous diffusion of scientific concepts.

By Thomas Maillart, Thibaut Chataing, David Dosu, Paul Bagourd, Julian Jang-Jaccard, Alain Mermoud

diffusion robotics benchmarks

arXiv Machine Learning

Jun 3

ReciNet: Reciprocal Space-Aware Long-Range Modeling for Crystalline Property Prediction

arXiv:2502. 02748v4 Announce Type: replace Abstract: Predicting properties of crystals from their structures is a fundamental yet challenging task in materials science.

By Jianan Nie, Peiyao Xiao, Kaiyi Ji, Peng Gao

benchmarks

arXiv AI

Jun 3

PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers

arXiv:2606. 03257v1 Announce Type: cross Abstract: Spiking Vision Transformer (SViT) models are promising low-power ViT models for solving vision-based tasks with state-of-the-art performance.

By Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio, Muhammad Shafique

llms computer-vision fine-tuning efficiency benchmarks

arXiv AI

Jun 3

EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents

arXiv:2606. 03678v1 Announce Type: new Abstract: Generating safety-critical scenarios is essential for validating and improving autonomous driving systems, yet it inherently requires maximizing adversariality to expose failures while preserving realism.

By Tong Nie, Yuewen Mei, Yihong Tang, Junlin He, Jie Deng, Jian Sun, Wei Ma

llms agents robotics benchmarks safety

arXiv Machine Learning

Jun 3

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606. 03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text.

By Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

arXiv:2606. 02581v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) faces a fundamental three-way tension: deeper retrieval improves factual grounding but inflates token costs and end-to-end latency.

By Sanjay Mishra

llms rag benchmarks safety

arXiv AI

Jun 3

Large AI Models in Dental Healthcare: From General-Purpose Systems to Domain-Specific Foundation Models

arXiv:2606. 02914v1 Announce Type: new Abstract: Background: Oral diseases affect nearly 3.

By Sema Helali, Lina Abu Nadab, Sausan Alqawas, Alaa Abd-Alrazaq, Faleh Tamimi, Rafat Damseh

agents computer-vision multimodal benchmarks safety

arXiv Machine Learning

Jun 3

Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

arXiv:2606. 03980v1 Announce Type: new Abstract: Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines.

By Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

arXiv:2606. 02596v1 Announce Type: new Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections).

By Anherutowa Calvo

llms benchmarks safety

arXiv AI

Jun 3

Relational Linearity is a Predictor of Hallucinations

arXiv:2601. 11429v2 Announce Type: replace-cross Abstract: Hallucination is a central failure mode of language models (LMs).

By Yuetian Lu, Yihong Liu, Sebastian Gerstner, Lea Hirlimann, Jonas Rohweder, Hinrich Sch\"utze

llms rag benchmarks safety

arXiv Machine Learning

Jun 3

Low-Frequency Shortcuts in Texture-Driven Visual Learning

arXiv:2606. 03493v1 Announce Type: cross Abstract: Neural networks suffer from shortcut learning, where learned features generalize well to the training set but not to in-distribution (ID) or out-of-distribution (OOD) test sets.

By Utku \c{S}irin, Cathy Hou, David Alvarez-Melis, Stratos Idreos

efficiency benchmarks

arXiv Machine Learning

Jun 3

A Cartesian-3j Framework for Machine Learning Interatomic Potentials

arXiv:2512. 16882v2 Announce Type: replace-cross Abstract: Machine learning interatomic potentials (MLIPs) have brought substantial gains in the extrapolation capability in computational chemistry.

By Zemin Xu, Chenyu Wu, Wenbo Xie, P. Hu

benchmarks safety

arXiv Machine Learning

Jun 3

ERP-XTTN: Interpretable Prototype-Guided Cross-Attention for Cross-Subject ERP Classification

arXiv:2606. 02939v1 Announce Type: new Abstract: Interpretable brain-computer interface classifiers that generalize across subjects without calibration remain an open challenge.

By Charlotte Genevier Wyman, Leanne Hirshfield

benchmarks safety

arXiv AI

Jun 3

Don't Gamble, GAMBLe: An Analytical Framework for AI-Driven Research Systems

arXiv:2606. 02863v1 Announce Type: new Abstract: AI-Driven Research Systems (ADRS) -- systems coupling LLMs with automated evaluation to discover algorithms, proofs, and designs -- are being optimized and adopted across domains, but the tools to analyze them have not kept pace.

By Marquita Ellis, Paul Castro

llms benchmarks

arXiv Machine Learning

Jun 3

Bayesian Tensor Decomposition with Diffusion Model Prior

arXiv:2606. 03212v1 Announce Type: new Abstract: Low-rank tensor decomposition (TD) is usually effective on clean, fully observed data, but it often degrades under severe missingness or noise.

By Zerui Tao, Qibin Zhao

diffusion efficiency benchmarks safety

arXiv Machine Learning

Jun 3

ArrowFlow: Hierarchical Machine Learning in the Space of Permutations

arXiv:2604. 04087v2 Announce Type: replace Abstract: We introduce ArrowFlow, a machine learning architecture that operates entirely in the space of permutations.

By Ozgur Yilmaz

efficiency benchmarks safety

arXiv AI

Jun 3

Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents

arXiv:2606. 03236v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have substantially advanced mobile agents, yet proactive mobile assistance remains challenging because agents must decide \emph{when} to intervene before determining \emph{how} to assist.

By Zhijie Ding (HyperAI Team, Xiaomi Corporation, Zhongnan University of Economics and Law), Weinan Hong (HyperAI Team, Xiaomi Corporation, Jilin University), Zicheng Zhu (HyperAI Team, Xiaomi Corporation, The Chinese University of Hong Kong, Shenzhen), Lei Li (HyperAI Team, Xiaomi Corporation), Dezhi Kong (HyperAI Team, Xiaomi Corporation), Hao Wang (HyperAI Team, Xiaomi Corporation), Peng Zhou (HyperAI Team, Xiaomi Corporation), Xuchu Jiang (HyperAI Team, Xiaomi Corporation), Jiaming Xu (HyperAI Team, Xiaomi Corporation)

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 3

Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension

arXiv:2405. 03386v2 Announce Type: replace Abstract: Training with noisy class labels impairs neural networks' generalization performance.

By Marek Herde, Lukas L\"uhrs, Denis Huseljic, Bernhard Sick

benchmarks

Get the daily briefing