Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Machine Learning

Sep 23

A Lightweight Plastic-Memory Framework for Graph Few-Shot Class-Incremental Learning

arXiv:2609.25781v1 Announce Type: new Abstract: Graph Incremental Learning has garnered increasing attention as dynamic graph data continues to emerge across diverse fields. Conventional approaches p...

By Zihan Mei, Zhili Qin, Tongze Zhang, Hongyuan Liu, Junming Shao, Qinli Yang

efficiency benchmarks

arXiv Machine Learning

Sep 23

Multi-View Fair Clustering Guided by Cross-View Sensitive Information Discrepancy

arXiv:2609.25811v1 Announce Type: new Abstract: Multi-view clustering (MVC) aims to uncover latent cluster structures by exploiting complementary information from multiple views. Despite substantial...

By Mudi Jiang, Jiahui Zhou, Xinying Liu, Zengyou He, Zhikui Chen

benchmarks safety

arXiv Machine Learning

Sep 23

Exploring Solver-Level Warmstarting for Neural Network Verification

arXiv:2609.25962v1 Announce Type: new Abstract: Neural network verification has become a key tool for providing formal guarantees on the behaviour of neural networks. However, many verification probl...

By Annelot Bosman, Minghao Liu, Marta Kwiatkowska, Holger Hoos, Jan van Rijn

benchmarks safety

arXiv Machine Learning

Sep 23

BOBA: Dynamic Bayesian Optimization through Bayesian Active Inference

arXiv:2609.26021v1 Announce Type: new Abstract: Dynamic black-box optimization presents significant challenges for Bayesian Optimization (BO), as the objective function evolves over time, causing opt...

By Merlin Angel Kelly, Rishan Patel, Alexander Thomas, Ziyue Zhu, Zikun Quan, Tom Carlson, Youngjun Cho

benchmarks

arXiv Machine Learning

Sep 23

Towards Adaptive Federated Graph Clustering: A Global Community-aware Contrastive Learning-based Approach

arXiv:2609.26063v1 Announce Type: new Abstract: Federated graph learning (FGL) enables multiple clients to collaboratively train graph models without sharing their private graph data, providing a pro...

By Yinlin Zhu, Di Wu, Wang Luo, Guocong Quan, Miao Hu

benchmarks

arXiv Machine Learning

Sep 23

CoEvo: Oracle-Grounded Self-Evolution of a Single Model for Multi-Step Causal Reasoning

arXiv:2609.26094v1 Announce Type: new Abstract: Multi-step causal reasoning requires chaining inferences where each step constrains the next. An early error propagates silently, and a correct answer...

By Jian Zhang, Bingyi Wang, Yizhi Liu

llms efficiency benchmarks safety

arXiv Machine Learning

Sep 23

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

arXiv:2609.26167v1 Announce Type: new Abstract: Activation-energy pruning -- removing weights whose product of magnitude and cumulative pre-synaptic spike count falls below a threshold -- was establi...

By Joseph Bingham

efficiency benchmarks

arXiv Machine Learning

Sep 23

Can You Delete a Year of Market Data? Machine Unlearning Against Exact Retraining Oracles

arXiv:2609.26242v1 Announce Type: new Abstract: When a data license expires, deleting stored records does not remove influence encoded in a trained forecaster. Machine unlearning seeks to remove this...

By Junyi Ye

llms benchmarks

arXiv Machine Learning

Sep 23

OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery

arXiv:2609.26402v1 Announce Type: new Abstract: The discovery of novel inorganic materials drives technological breakthroughs in critical fields such as computing and energy storage. Generative AI ha...

By Thomas Egg, Harry Winston Sullivan, Ellad B. Tadmor, Stefano Martiniani

diffusion fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 23

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

arXiv:2609.26631v1 Announce Type: new Abstract: Accurate ground-based cloud classification is important for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate...

By Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw Zegarlinski

computer-vision fine-tuning benchmarks

arXiv Machine Learning

Sep 23

SPARC: SuperPixel-Aware Region Contrastive Learning for Self-Supervised Dense Prediction

arXiv:2609.25067v1 Announce Type: cross Abstract: Self-supervised learning (SSL) has become an effective approach for learning visual representations without manual annotations. Among SSL approaches,...

By David Szczecina, Yuanpei Xiang, Jitao Hu, David Clausi, Yuhao Chen, Jason Deglint, Paul Fieguth

computer-vision benchmarks

arXiv Machine Learning

Sep 23

Calibration Count Reuse: Validity Does Not Determine Efficiency

arXiv:2609.25138v1 Announce Type: cross Abstract: Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity sco...

By Rudra Chopra

benchmarks

arXiv Machine Learning

Sep 23

Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Study

arXiv:2609.25978v1 Announce Type: cross Abstract: Saliency maps are widely used to explain deep learning predictions in medical imaging, yet visually plausible explanations do not necessarily reflect...

By Peachapong Poolpol, Henrik H. J. Detjen, Eike Petersen

llms computer-vision benchmarks

arXiv Machine Learning

Sep 23

RankCert: When Can Simulated Learners Safely Select an AI Tutor? Robust Decision Certification Under Structural Uncertainty

arXiv:2609.26069v1 Announce Type: cross Abstract: Simulation-based tutor selection can be unstable when predictively adequate learner models imply different policy rankings. RankCert certifies one of...

By Nizam Kadir

benchmarks safety

arXiv Machine Learning

Sep 23

Towards Mitigating Excessive Forgetting in LLM Unlearning via Entanglement-Guidance with Proxy Constraint

arXiv:2508.20443v3 Announce Type: replace Abstract: Large language models (LLMs) are trained on massive datasets that may include private or copyrighted content. Due to growing privacy and ownership...

By Zhihao Liu, Jian Lou, Yuke Hu, Xiaochen Li, Yitian Chen, Tailun Chen, Zhizhen Qin, Kui Ren, Zhan Qin

llms benchmarks safety

arXiv Machine Learning

Sep 23

Refit the Probe: Single-Direction Ablation Is Not a Necessity Test

arXiv:2606.00926v2 Announce Type: replace Abstract: Probes are routinely paired with an intervention: ablate the direction the probe found, run the model, and read the change in task accuracy, taking...

By Yuhang Jiang

benchmarks

arXiv Machine Learning

Sep 23

Glucose-ML: A collection of longitudinal diabetes datasets for development of robust AI solutions

arXiv:2507.14077v2 Announce Type: replace-cross Abstract: Artificial intelligence (AI) algorithms are a critical part of state-of-the-art digital health technology for diabetes management. Yet, acces...

By Temiloluwa Prioleau, Baiying Lu, Yanjun Cui

benchmarks

arXiv Machine Learning

Sep 23

VeriSoftBench: Repository-Scale Formal Verification Benchmarks for Lean

arXiv:2602.18307v2 Announce Type: replace-cross Abstract: Large language models have achieved striking results in interactive theorem proving, particularly in Lean. However, most benchmarks for LLM-b...

By Yutong Xin, Qiaochu Chen, Greg Durrett, I\c{s}il Dillig

llms benchmarks

arXiv Machine Learning

Sep 23

LLM Ghostbusters: Surgical Package Hallucination Suppression via Adaptive Unlearning

arXiv:2605.01047v2 Announce Type: replace-cross Abstract: Hallucinations remain an unsolved problem for LLMs, and package hallucinations are a particularly dangerous instance of this phenomenon. Pack...

By Joseph Spracklen, Pedram Aghazadeh, Farinaz Koushanfar, Murtuza Jadliwala

llms agents benchmarks safety

arXiv Computation and Language

Sep 23

AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search

arXiv:2609.25047v1 Announce Type: new Abstract: Autonomous agents that automatically build artificial intelligence (AI) models could broaden access to AI across science and engineering. A popular lin...

By Peijia Qin, Ruiyi Zhang, Qi Cao, Han Guo, Li Zhang, Pengtao Xie

llms agents benchmarks

Get the daily briefing