Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Machine Learning

Sep 30

Best Practices in EEG Analysis: Preprocessing, Modeling, and Machine Learning

arXiv:2609.36609v1 Announce Type: cross Abstract: Electroencephalography (EEG) analysis requires careful choices in preprocessing, statistical modeling, and machine learning because EEG signals are h...

By Parsa Razmara, Woojae Jeong, Aditya Kommineni, Raymundo Cassani, Richard Leahy, Takfarinas Medani

benchmarks

arXiv Machine Learning

Sep 30

Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

arXiv:2609.36798v1 Announce Type: cross Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training para...

By Yueran Ma, Ronghao Lin

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

You Cannot Recover What Was Never Measured: Quantifying the Information Ceiling of Ultra-Low-Field MRI Super-Resolution

arXiv:2609.36837v1 Announce Type: cross Abstract: Generative super-resolution models can turn portable 64 mT MRI into images that look like 3T scans, and the field evaluates them with PSNR, SSIM, and...

By Prathamesh Pradeep Khole, Shreya Handa, Utkarsh Gupta, Razvan Marinescu

llms diffusion benchmarks

arXiv Machine Learning

Sep 30

RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning

arXiv:2609.36885v1 Announce Type: cross Abstract: RNA design aims to identify sequences that fold into specified secondary structures. Existing methods formulate the task as target-specific search or...

By Zefeng Lin, Xianyong Fang, Tianfan Fu, Xiaohua Xu

diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

GleanVID: Complementary Token Selection for Efficient Video Large Language Models

arXiv:2609.37042v1 Announce Type: cross Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the lar...

By Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang, Baochang Zhang

llms benchmarks

arXiv AI

Sep 30

Follow the Entities: A Corpus Map for Agentic Search

arXiv:2609.37226v1 Announce Type: cross Abstract: Answering questions and completing tasks over large document collections often requires connecting evidence spread across multiple documents, such as...

By Soyeong Jeong, Sujay Kumar Jauhar, Sung Ju Hwang, Andrew Joohun Nam

llms agents benchmarks

arXiv Machine Learning

Sep 30

SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems

arXiv:2609.37294v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks, yet their stochastic behavior challenges the validity, reproducibi...

By Francisco Ortin

llms benchmarks

arXiv AI

Sep 30

MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller

arXiv:2609.37304v1 Announce Type: new Abstract: Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not...

By Zhibin Wen, Tao Han, Lei Bai, Can Li, Yang Xu

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG

arXiv:2609.37469v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without...

By Suting Chen, Peichun Hua, Yunming Xiao

llms rag benchmarks

arXiv Machine Learning

Sep 30

Physics-Guided Flow-Map Matching for Precipitation Nowcasting

arXiv:2609.37487v1 Announce Type: cross Abstract: Precipitation nowcasting, generating future radar fields from past observations, is critical for flood warning and disaster response. It is also a de...

By Shunya Nagashima, Takumi Bannai, Makoto Misaizu, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

benchmarks

arXiv Machine Learning

Sep 30

From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation

arXiv:2609.37510v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own reasoning trajectories using feedback from a stronger teacher. Teacher interventions can imp...

By Yuhao Wang, Ruiyang Ren, Yinan Zhang, Ruiqing Zhang, Jing Liu, Chunyan Miao

agents efficiency benchmarks

arXiv Machine Learning

Sep 30

Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study

arXiv:2609.37848v1 Announce Type: cross Abstract: Biomedical machine learning papers often compress model performance into one headline number. That number can look like a property of the model even...

By Bhanu Prakash Vangala, Sowmya Guda, Latha Peddi, Navya Vangala

fine-tuning benchmarks

arXiv Machine Learning

Sep 30

How Many Labels Does a Language Need? Annotation Budgets and Cross-Lingual Pooling for African-Language Text Classification

arXiv:2609.37882v1 Announce Type: cross Abstract: Every text classifier for an African language begins with a budgeting question: how many labelled examples are needed, and can labels from other Afri...

By Bhanu Prakash Vangala, Sowmya Guda, Navya Vangala

llms nlp benchmarks

arXiv Machine Learning

Sep 30

ReCAP: Retrieval-Guided Capability Reuse for Multimodal Continual Instruction Tuning

arXiv:2609.37889v1 Announce Type: cross Abstract: Multimodal continual instruction tuning (MCIT) aims to enable multimodal large language models to acquire new capabilities from sequential tasks whil...

By Tao Hu, Zhinuo Zhou, Xialiang Tong, De-Chuan Zhan, Da-Wei Zhou

llms rag fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs

arXiv:2609.37891v1 Announce Type: cross Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...

By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko

llms benchmarks

arXiv Machine Learning

Sep 30

Latent Inference-Time Guidance of Time Series Foundation Models

arXiv:2609.38058v1 Announce Type: cross Abstract: Time Series Foundation Models (TSFMs) currently provide state-of-the-art results in forecasting tasks. They are available out-of-the-box and rely on...

By Chlo\'e Hashimoto-Cullen, Amaury Durand, Laurent Bozzi, Benjamin Guedj, Yannig Goude, Sylvain Le Corff

llms diffusion benchmarks

arXiv Machine Learning

Sep 30

Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data

arXiv:2609.38165v1 Announce Type: cross Abstract: The landscape of satellite imagery time series datasets and boundary-pushing architectures for cropland segmentation has never been richer. However,...

By Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini

llms computer-vision benchmarks

arXiv AI

Sep 30

STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

arXiv:2609.38169v1 Announce Type: cross Abstract: Linear attention replaces growing KV caches with fixed-size recurrent states, yet these persistent states can become a substantial memory bottleneck...

By Bingchen Yao, Haobo Xu, Haokun Lin, Yichen Wu, Ziyu Guo, Renrui Zhang, Zhichao Lu, Zhenan Sun, Ying Wei

efficiency benchmarks

arXiv Machine Learning

Sep 30

Quantum Geometry of Data

arXiv:2507.21135v2 Announce Type: replace Abstract: We demonstrate how Quantum Cognition Machine Learning (QCML) encodes data as quantum geometry. In QCML, features of the data are represented by lea...

By Alexander G. Abanov, Luca Candelori, Harold C. Steinacker, Martin T. Wells, Jerome R. Busemeyer, Cameron J. Hogan, Vahagn Kirakosyan, Nicola Marzari, Sunil Pinnamaneni, Dario Villani, Mengjia Xu, Kharen Musaelian

rag benchmarks

arXiv Machine Learning

Sep 30

Data-Efficient Time-Dependent PDE Surrogates: Graph Neural Simulators vs. Neural Operators

arXiv:2509.06154v3 Announce Type: replace Abstract: Developing accurate, data-efficient surrogate models is central to advancing AI for Science. Neural operators (NOs), which approximate mappings bet...

By Dibyajyoti Nayak, Somdatta Goswami

benchmarks

Get the daily briefing