Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,494 stories · RSS feed

arXiv AI

Jun 3

Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning

arXiv:2606. 03762v1 Announce Type: cross Abstract: Agentic reinforcement learning (RL) equips large language models (LLMs) with tool-use capabilities that substantially improve reasoning on complex tasks.

By Hongye Cao, Nuo Yan, Haoyuan Deng, Ziwei Wang, Tianpei Yang, Jing Huo, Yuyao Zhang, Yang Gao

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 3

The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol

arXiv:2606. 03907v1 Announce Type: cross Abstract: Agentic AI coding tools write code with increasing autonomy and in doing so decide when to import a library and when to implement functionality from scratch.

By Jai Lal Lulla, Matthias Galster, Jie M. Zhang, Sebastian Baltes, Christoph Treude

llms agents benchmarks

arXiv Machine Learning

Jun 3

Learning Unmasking Policies for Diffusion Language Models

arXiv:2512. 09106v4 Announce Type: replace Abstract: Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference.

By Metod Jazbec, Theo X. Olausson, Louis B\'ethune, Pierre Ablin, Michael Kirchhof, Jo\~ao Monteiro, Victor Turrisi, Jason Ramapuram, Marco Cuturi

llms diffusion reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 3

Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)

arXiv:2605. 18629v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are one of the main methods to interpret the inner workings of deep neural networks (DNNs), decomposing activations into higher-dimensional features.

By Micha{\l} Brzozowski, Neo Christopher Chung

efficiency benchmarks safety

arXiv Machine Learning

Jun 3

TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer

arXiv:2506. 00431v2 Announce Type: replace Abstract: Due to the proficiency of self-attention mechanisms (SAMs) in capturing dependencies in sequence modeling, several existing dynamic graph neural networks (DGNNs) utilize Transformer architectures with various encoding designs to capture sequential evolutions of dynamic graphs.

By Jie Peng, Zhewei Wei, Yuhang Ye

llms rag benchmarks safety

arXiv AI

Jun 3

Relational Linearity is a Predictor of Hallucinations

arXiv:2601. 11429v2 Announce Type: replace-cross Abstract: Hallucination is a central failure mode of language models (LMs).

By Yuetian Lu, Yihong Liu, Sebastian Gerstner, Lea Hirlimann, Jonas Rohweder, Hinrich Sch\"utze

llms rag benchmarks safety

arXiv Machine Learning

Jun 3

Low-Frequency Shortcuts in Texture-Driven Visual Learning

arXiv:2606. 03493v1 Announce Type: cross Abstract: Neural networks suffer from shortcut learning, where learned features generalize well to the training set but not to in-distribution (ID) or out-of-distribution (OOD) test sets.

By Utku \c{S}irin, Cathy Hou, David Alvarez-Melis, Stratos Idreos

efficiency benchmarks

arXiv Machine Learning

Jun 3

CoughSense: Five-Class Respiratory Disease Classification via Whisper Encoder Fine-Tuning and Dual-Encoder Cross-Attention Fusion with Balanced Contrastive Learning

arXiv:2606. 02998v1 Announce Type: new Abstract: Automated cough analysis offers a path to low-cost respiratory screening, but most existing work stops at binary COVID-19 detection.

By Nikhil Vincent

fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 3

SAHG: Sector-Anisotropic Hyperbolic Graph Model for Social Bot Detection

arXiv:2605. 30166v2 Announce Type: replace-cross Abstract: LLM-driven social bots can generate fluent, human-like text, reducing the discriminative advantage of content-based detection alone.

By Hanning Lu, Yingguang Yang, Jinwei Su, Yang Liu, Zhaoqian Yao, Yaoming Li, Taoran Liang, Ziyi Zhang, Ran Ran, Kefu Xu, Bin Chong

llms benchmarks safety

arXiv Machine Learning

Jun 3

WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents

arXiv:2605. 20306v2 Announce Type: replace-cross Abstract: We introduce WildRoadBench, a wild aerial road-damage grounding benchmark that couples direct visual grounding by vision-language models with autonomous research-and-engineering by LLM-driven agents on a single professionally annotated UAV corpus.

By Bingnan Liu, Chenhang Cui, Rui Huang, Jiani Luo, Zhirong Shen, Tinghao Wang, Xiande Huang, Lingbei Meng, Fei Shen, An Zhang

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 3

LC-SAC: Lyapunov-Constrained Soft Actor-Critic via Koopman Operator Theory for Trajectory Tracking and Stabilization

arXiv:2602. 04132v4 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) has achieved remarkable success in solving complex sequential decision-making problems.

By Dhruv S. Kushwaha, Zoleikha A. Biron

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 3

Towards Blind Lens Aberration Correction via Large LensLib Pre-training and Discrete Degradation Priors

arXiv:2511. 17126v4 Announce Type: replace-cross Abstract: Emerging deep-learning-based lens library pre-training (LensLib-PT) pipeline offers a new avenue for blind lens aberration correction by training a universal neural network, demonstrating strong capability in handling diverse unknown optical degradations.

By Xiaolong Qian, Qi Jiang, Yao Gao, Lei Sun, Kailun Yang, Xian Wang, Zhonghua Yi, Wenyong Li, Ming-Hsuan Yang, Luc Van Gool, Kaiwei Wang

efficiency benchmarks

arXiv Machine Learning

Jun 3

ReciNet: Reciprocal Space-Aware Long-Range Modeling for Crystalline Property Prediction

arXiv:2502. 02748v4 Announce Type: replace Abstract: Predicting properties of crystals from their structures is a fundamental yet challenging task in materials science.

By Jianan Nie, Peiyao Xiao, Kaiyi Ji, Peng Gao

benchmarks

arXiv Machine Learning

Jun 3

Forecasting Conceptual Diffusion in Science: The Case of Quantum Computing

arXiv:2606. 03919v1 Announce Type: cross Abstract: Understanding and anticipating scientific change requires models that distinguish between endogenous consolidation and exogenous diffusion of scientific concepts.

By Thomas Maillart, Thibaut Chataing, David Dosu, Paul Bagourd, Julian Jang-Jaccard, Alain Mermoud

diffusion robotics benchmarks

arXiv Machine Learning

Jun 3

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606. 03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text.

By Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

llms rag fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 3

Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning

arXiv:2606. 03234v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has become the dominant approach for improving mathematical reasoning in large language models, yet current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.

By Ziyue Wang, Aomufei Yuan, Yongfu Zhu, Shuai Dong, Wenpu Liu, Yiran Yao, Weichu Xie, Yuqi Xu, Caoyuan Ma, Wenqi Shao, Xiaoying Zhang, Nan Duan, Jiaqi Wang

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 3

Learning Temporal Causal Structure via Smooth Differentiable Optimization

arXiv:2606. 03227v1 Announce Type: new Abstract: Causal discovery with instantaneous effects in multivariate time series is challenging, as the instantaneous structure must be acyclic.

By Tong Zhao, Ce Guo, Wayne Luk, Emil Lupu, Ray Dipojjwal

benchmarks

arXiv Machine Learning

Jun 3

Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation

arXiv:2606. 03130v1 Announce Type: new Abstract: Small open-source code models that power IDE autocomplete still emit hallucinated Fill-in-the-Middle (FIM) completions: syntactically natural calls to methods, parameters, variables, and imports that do not exist in the surrounding project.

By Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg, Amabel Gale, Xiaoyu Liu, Pareesa Ameneh Golnari, Shengyu Fu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 3

Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching

arXiv:2606. 03199v1 Announce Type: new Abstract: Organic crystal structure prediction (CSP) is a requirement for computational modelling of organic solids, but traditionally costs several CPU-years per molecule.

By Alston Lo, Luka Mucko, Austin H. Cheng, Andy Cai, Alastair J. A. Price, Wojciech Matusik, Al\'an Aspuru-Guzik

diffusion benchmarks safety

arXiv Machine Learning

Jun 3

Will Accurate Fields Mislead Photonic Design? FromGlobal Accuracy to Port Readout

arXiv:2606. 03038v1 Announce Type: new Abstract: Neural field surrogates can accelerate photonic design loops, but a surrogate that looks accurate in global field error can still mis-rank candidate devices when the final decision depends on localized output-port readouts.

By Yitian Zhang, Yonghong chen, Youming Chen, Yiyang Li, Xing Zhe, Renhe Lu, Shaolin Liao, Yuzhe Ma, Zhong Guan

benchmarks

Get the daily briefing