Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 5

EML-CD: Causal Mechanism Recovery via EML Symbolic Trees in Structure Learning

arXiv:2606. 05942v1 Announce Type: cross Abstract: Neural network (NN)-based nonlinear causal discovery methods recover DAG structure but leave each causal mechanism as a black box.

By Sota Asanuma

benchmarks

arXiv Machine Learning

Jun 5

Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning

arXiv:2606. 05675v1 Announce Type: new Abstract: Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge.

By Hongye Xu, Bartosz Krawczyk

rag benchmarks safety

arXiv Machine Learning

Jun 5

An accurate nucleic acid-small molecule docking framework via geometric deep learning with large-scale pretraining

arXiv:2606. 05198v1 Announce Type: cross Abstract: Nucleic acids are increasingly recognized as therapeutic targets beyond conventional protein-centered drug discovery, yet accurate and efficient docking of small molecules to nucleic acid structures remains challenging.

By Shi Li (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China), Xujun Zhang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China), Mingquan Liu (Faculty of Health Sciences, University of Macau, Macau SAR, China), Hui Zhang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Shuoying Jia (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Yu Kang (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Shanghai Innovation Institute, Shanghai, China), Tingjun Hou (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Zhejiang Provincial Key Laboratory for Intelligent Drug Discovery and Development, Jinhua Institute of Zhejiang University, Zhejiang, China), Peichen Pan (College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang, P. R. China, Zhejiang Provincial Key Laboratory for Intelligent Drug Discovery and Development, Jinhua Institute of Zhejiang University, Zhejiang, China)

fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

arXiv:2606. 05516v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distributed across layers.

By Wanhao Yu, Ziyan Wang, Zheng Wang, Abeer Matar Almalky, Yihang Zuo, Shuteng Niu, Sen Lin, Adnan Siraj Rakin, Deliang Fan, Li Yang

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs

arXiv:2606. 05376v1 Announce Type: new Abstract: Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators.

By Jingyao Wu, Ashley Wang, Keane Ong, Paul Pu Liang, Rosalind Picard

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 5

ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

arXiv:2606. 05170v1 Announce Type: new Abstract: At matched accuracy, open-weight LLMs differ substantially in the shape of their error severity distribution -- a difference invisible to the scalar error rate.

By Jason Z Wang

llms benchmarks safety

arXiv Machine Learning

Jun 5

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.

By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Query-efficient model evaluation using cached responses

arXiv:2605. 07096v2 Announce Type: replace Abstract: Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment.

By Hayden Helm, Ben Johnson, Carey Priebe

benchmarks

arXiv Machine Learning

Jun 5

Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction

arXiv:2606. 06303v1 Announce Type: new Abstract: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining.

By Hongkun Dou, Zike Chen, Fengji Li, Hongjue Li, Yue Deng

diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution

arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.

By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang

rag benchmarks

arXiv Machine Learning

Jun 5

A Survey on Diffusion Language Models

arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.

By Tianyi Li, Mingda Chen, Bowei Guo, Zhiqiang Shen

llms diffusion nlp efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

Critic-Guided Heterogeneous Multi-Agent Reasoning for Reliable Mathematical Problem Solving

arXiv:2606. 05704v1 Announce Type: cross Abstract: Recent Large Language Models (LLMs) have shown impressive reasoning abilities; but they are still susceptible to hallucinations, intermediate reasoning mistakes, and unreliable reasoning results in complex mathematical reasoning problems.

By Muhammad Talha Sharif, Abdul Rehman

llms agents benchmarks safety

arXiv Machine Learning

Jun 5

Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models

arXiv:2606. 05378v1 Announce Type: new Abstract: We test whether a single screen-and-ablate recipe -- identify attention-head circuits by task-pattern selectivity, then verify by causal ablation against a matched-random null -- produces consistent mechanistic claims across model families.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Jun 5

Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models

arXiv:2606. 05434v1 Announce Type: new Abstract: Group Relative Policy Optimisation (GRPO) has emerged as an effective reinforcement-learning algorithm for aligning language models on reasoning tasks, but it treats every token position and every sampled rollout symmetrically.

By Chirag Chawla, Rohan Charudatt Salvi, Madhav S. Baidya

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 5

Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models

arXiv:2606. 05531v1 Announce Type: cross Abstract: Despite the rapid progress of Vision-Language Models (VLMs), the field lacks benchmarks that rigorously diagnose their true reasoning abilities and chart meaningful progress toward human-like multimodal intelligence.

By Mohammad Mahdi Abootorabi, Omid Ghahroodi, Anas Madkoor, Marzia Nouri, Doratossadat Dastgheib, Mohamed Hefeeda, Ehsaneddin Asgari

llms multimodal benchmarks

arXiv Machine Learning

Jun 5

Balancing Image Compression and Generation with Bootstrapped Tokenization

arXiv:2606. 05552v1 Announce Type: new Abstract: Despite progress in image tokenization, standard methods encode redundant information by mixing all granularities within each token, thus redundancy persists between tokens.

By Haozhe Chi, Jinghan Li, Hao Jiang, Wu Sheng, Yi Ma, Jing Wang, Yadong Mu

nlp benchmarks

arXiv Machine Learning

Jun 5

HDST-GNN: Heterogeneous Dynamic Spatiotemporal Graph Neural Networks for Multi-Object Tracking in UAV Aerial Imagery

arXiv:2606. 05587v1 Announce Type: cross Abstract: Multi-object tracking (MOT) from UAV imagery presents unique challenges: altitude varies across sequences, objects are small and densely packed, and frequent occlusion causes identity switches.

By Phillip Jiang

rag benchmarks

arXiv Machine Learning

Jun 5

Stochastic-Dimension Frozen Sampled Neural Network for High-Dimensional Gross-Pitaevskii Equations on Unbounded Domains

arXiv:2604. 09361v3 Announce Type: replace Abstract: This paper introduces the Stochastic-Dimension Frozen Sampled Neural Network (SD-FSNN), a novel computational framework for solving high-dimensional Gross-Pitaevskii equation (GPE) on unbounded domain.

By Zhangyong Liang, Tingfeng Wang, Xiaofei Zhao

benchmarks

arXiv Machine Learning

Jun 5

Know Yourself Better: Diverse Object-Related Features Improve Open Set Recognition

arXiv:2404. 10370v4 Announce Type: replace-cross Abstract: Open set recognition (OSR) is a critical aspect of machine learning, addressing the challenge of detecting novel classes during inference.

By Jiawen Xu, Margret Keuper

benchmarks

Get the daily briefing