Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,494 stories · RSS feed

arXiv Machine Learning

Jun 3

Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals

arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.

By Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang, Xinpei Wang, Weitong Chen

multimodal benchmarks

arXiv Machine Learning

Jun 3

RRISE: Robust Radius Inference via a Surrogate Estimator

arXiv:2606. 02876v1 Announce Type: new Abstract: Randomized smoothing (RS) uses a smoothed classifier to provide architecture-agnostic certificates of $\ell_2$ classification robustness, but its dependence on per-input Monte Carlo (MC) sampling undermines its use in real-time systems.

By Jong-Ik Park, Shreyas Chaudhari, Carlee Joe-Wong, Jos\'e M. F. Moura

computer-vision benchmarks

arXiv AI

Jun 3

Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs

arXiv:2606. 03879v1 Announce Type: cross Abstract: As foundation models scale toward fusing more heterogeneous visual streams, understanding how diverse encoders interact under joint training becomes a prerequisite for principled design.

By Wei Ding, Yudong Zhang, Ruobing Xie, Xingwu Sun, Jiansheng Chen, Yu Wang

llms multimodal benchmarks

arXiv AI

Jun 3

SL-BiLEM: Structured Learnable Behavior-in-the-Loop Epidemic Modeling for Forecasting and Policy Evaluation

arXiv:2605. 26704v2 Announce Type: replace-cross Abstract: Epidemic forecasting faces a fundamental challenge: human behavior dynamically responds to disease spread, creating feedback loops that induce distribution shifts at policy intervention points.

By Haochun Wang, Sendong Zhao, Jingbo Wang, Yanrui Du, Ting Liu, Bing Qin

benchmarks

arXiv AI

Jun 3

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

arXiv:2507. 19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities.

By Bermet Burkanova, Yasaman Etesam, Payam Jome Yazdian, Trinity Evans, Chuxuan Zhang, Zoe Stanley, Paige Tutt\"os\'i, Angelica Lim

llms robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

UR$^2$: Unify RAG and Reasoning through Reinforcement Learning

arXiv:2508. 06165v5 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown strong capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG) for knowledge grounding and Reinforcement Learning from Verifiable Rewards (RLVR) for complex reasoning.

By Weitao Li, Boran Xiang, Xiaolong Wang, Zhinan Gou, Weizhi Ma, Yang Liu

llms rag reinforcement-learning benchmarks

arXiv AI

Jun 3

FFR: Forward-Forward Learning for Regression

arXiv:2606. 03927v1 Announce Type: cross Abstract: The Forward-Forward (FF) algorithm offers a computationally efficient and biologically plausible alternative to backpropagation (BP) by training neural networks through purely local, layer-wise optimization.

By Xinyang Liu, Xuanyu Liang, Shiqi Ding, Boyang Li, Zhiqiang Que, Jiayang Li, Guosheng Hu

benchmarks

arXiv Machine Learning

Jun 3

Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs

arXiv:2606. 02628v1 Announce Type: new Abstract: We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest.

By Aizierjiang Aiersilan

llms efficiency benchmarks safety

arXiv AI

Jun 3

ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models

arXiv:2606. 03157v1 Announce Type: new Abstract: Large language models (LLMs) have been widely adopted in healthcare, yet they still encounter significant challenges in complex clinical decision-making scenarios.

By Ruihui Hou, Siyi Zhu, Ziyue Huai, Guangya Yu, Yongqi Fan, Chunming Wang, Tong Ruan

llms agents benchmarks

arXiv AI

Jun 3

What Makes Interaction Trajectories Effective for Training Terminal Agents?

arXiv:2606. 03461v1 Announce Type: new Abstract: Stronger code agents are commonly assumed to be superior teachers for post-training, yet this assumption remains poorly disentangled from task difficulty, harness design, and student capacity.

By Sidi Yang, Chaofan Tao, Jierun Chen, Tiezheng Yu, Ruoyu Wang, Yuxin Jiang, Yiming Du, Wendong Xu, Jing Xiong, Taiqiang Wu, Lifeng Shang, Xiaohui Li, Ngai Wong, Haoli Bai

llms agents fine-tuning benchmarks

arXiv AI

Jun 3

ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning

arXiv:2606. 03503v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) have achieved remarkable progress thanks to Reinforcement Learning with Verifiable Rewards (RLVR) on Chain-of-Thoughts (CoTs).

By Ziyan Liu, Xueda Shen, Yuzhe Gu, Songyang Gao, Kuikun Liu, Guangran Cheng, Chengqi Lyu, Dahua Lin, Wenwei Zhang, Kai Chen

reinforcement-learning benchmarks

arXiv AI

Jun 3

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

arXiv:2606. 03657v1 Announce Type: new Abstract: Large language models for code generation often need to use APIs that are absent from their pretraining data.

By Jinnuo Liu, Yue Peng, Jinhan Niu, Hongyi Wen

llms agents fine-tuning benchmarks

arXiv AI

Jun 3

Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs

arXiv:2606. 03705v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are widely used to mitigate the limitations of Large Language Models (LLMs), such as outdated knowledge and hallucinations.

By Weiwei Ding, Zixuan Li, Long Bai, Zhuo Chen, Kun Su, Fei Wang, Xiaolong Jin, Jin Zhang, Jiafeng Guo, Xueqi Cheng

llms benchmarks safety

arXiv Machine Learning

Jun 3

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

arXiv:2606. 02596v1 Announce Type: new Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections).

By Anherutowa Calvo

llms benchmarks safety

arXiv Machine Learning

Jun 3

RESCAST-100K: A Comprehensive Dataset for Cross-Domain Residential Load and Indoor Temperature Forecasting

arXiv:2606. 02852v1 Announce Type: new Abstract: Accurate short-term forecasting of residential energy load and indoor temperature is essential for home energy management systems, grid-level demand response, and community energy efficiency efforts.

By Jainam Dhruva, Yousaf Raza, A. B. Siddique, Simone Silvestri

llms robotics fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Parallel Adaptive Multi-Objective Evolutionary Learning of Discretized Bayesian Network Classifiers for Clinical Data

arXiv:2605. 29058v2 Announce Type: replace Abstract: Bayesian Networks (BNs) are of interest from an explainable AI viewpoint, offering transparent probabilistic models for decision support.

By Damy M. F. Ha, Thalea Schlender, Yvette M. van der Linden, Peter A. N. Bosman, Tanja Alderliesten

benchmarks

arXiv Machine Learning

Jun 3

Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

arXiv:2606. 03980v1 Announce Type: new Abstract: Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines.

By Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 3

InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning

arXiv:2602. 06960v3 Announce Type: replace-cross Abstract: Large reasoning models achieve strong performance by scaling inference-time chain-of-thought, but this paradigm suffers from quadratic cost, context length limits, and degraded reasoning due to lost-in-the-middle effects.

By Yuchen Yan, Liang Jiang, Jin Jiang, Shuaicheng Li, Zujie Wen, Zhiqiang Zhang, Jun Zhou, Jian Shao, Yueting Zhuang, Yongliang Shen

nlp reinforcement-learning efficiency benchmarks

arXiv AI

Jun 3

Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation

arXiv:2606. 03128v1 Announce Type: cross Abstract: Smart contracts face critical security challenges that require thorough auditing in decentralized web services.

By Bagus Rakadyanto Oktavianto Putra, Muhamad Risqi Utama Saputra, Widyawan, Guntur Dharma Putra

llms efficiency benchmarks safety

arXiv AI

Jun 3

Toward Training Superintelligent Software Agents through Self-Play SWE-RL

arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.

By Yuxiang Wei, Zhiqing Sun, Emily McMilin, Jonas Gehring, David Zhang, Gabriel Synnaeve, Daniel Fried, Lingming Zhang, Sida Wang

llms agents reinforcement-learning benchmarks

Get the daily briefing