Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

15,528 stories · RSS feed

arXiv Machine Learning

Aug 3

Learning Optimal Dynamic Matching via Graph Neural Networks

arXiv:2607. 28925v1 Announce Type: new Abstract: Dynamic matching markets require decisions about whom to match and when: matching now yields value but removes participants who may create better future opportunities.

By Genta Okada, Shunya Noda, Junpei Komiyama, Akira Matsushita

benchmarks

arXiv Machine Learning

Aug 3

Mining Verdict Boundaries for Neural Network Verification

arXiv:2607. 28954v1 Announce Type: new Abstract: Branch and Bound (BaB) aims to achieve complete verification of neural networks by adaptively partitioning the problem and applying off-the-shelf verifiers to subproblems.

By Jiawei Ren, Guanqin Zhang, Zhenya Zhang, Yulei Sui

benchmarks

arXiv Machine Learning

Aug 3

Dynamics-aware identification of governing equations from sparse and noisy data

arXiv:2607. 29036v1 Announce Type: new Abstract: Sparse identification of nonlinear dynamics (SINDy) and PDE functional identification (PDE-FIND) recover parsimonious ordinary and partial differential equations (ODEs and PDEs) from data.

By Pongpisit Thanasutives, Yoshinobu Kawahara

diffusion benchmarks

arXiv Machine Learning

Aug 3

What Is Missing in Surgical Risk Stratification and Outcome Prediction: A Scoping Review of End-to-End Machine Learning Approaches

arXiv:2607. 29090v1 Announce Type: new Abstract: Postoperative adverse events, including mortality and morbidity, remain a major global burden, many of which are preventable through early identification of high-risk patients and targeted perioperative care.

By Yizhi Dong, Yuhe Ke, Hairil Rizal Abdullah, Yucheng Xing, Kevan Kai Bing Teo, Ling Huang, Mengling Feng

multimodal benchmarks

arXiv Machine Learning

Aug 3

Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

arXiv:2607. 29225v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance.

By Panagiotis Krokidas, Christoforos Rekatsinas, Vassilis Sioros, Grigorios M. Chatziathanasiou, Efi-Maria Papia, George Giannakopoulos

robotics benchmarks

arXiv Machine Learning

Aug 3

Assessing the Generalization of Graph Neural Networks for Fault Location Across Increasing Distributed Energy Resource Penetration Levels

arXiv:2607. 29293v1 Announce Type: new Abstract: Accurate fault location is critical for distribution network reliability.

By Burak Karabulut, Olayiwola Arowolo, Carlo Manna, Chris Develder, Jochen L. Cremer

benchmarks

arXiv Machine Learning

Aug 3

Conditioning Tree-Based Diffusions and Flows for Probabilistic Tabular Regression

arXiv:2607. 28864v1 Announce Type: cross Abstract: Tree-based diffusion models fit flexible conditional predictive distributions for tabular regression without a neural density estimator, but they inherit their design defaults---noising path, parameterization, training distribution, features, sampler---from the neural setting.

By Silas Koemen

diffusion benchmarks

arXiv Machine Learning

Aug 3

Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair

arXiv:2607. 28877v1 Announce Type: cross Abstract: Verification consumes the majority of modern chip design effort, yet the formal verification tools that provide mathematical guarantees of correctness remain expensive and restrictively licensed.

By Ha Trung Tran

llms agents benchmarks

arXiv Machine Learning

Aug 3

SciFigPlag-Bench: A Benchmark for Provenance-Aware Scientific Figure Plagiarism Detection

arXiv:2607. 29124v1 Announce Type: cross Abstract: Scientific figures often encode the visual evidence behind scientific findings, yet figure plagiarism remains underexplored as a benchmarked multimodal evaluation problem.

By Zhiying Cui, Minghao Yang, Linlin Gao, Jie Liu, Pengyuan Li

llms multimodal benchmarks

arXiv Machine Learning

Aug 3

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

arXiv:2607. 29213v1 Announce Type: cross Abstract: Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent.

By Jiping Liu, Zhongmin Zhang, Zisen Sang, Zhijia Fang, Tao Ouyang, Ma Jiang, Shaopeng Liang, Zeyang Hou, Guodong Cao, Jia Jia

rag reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 3

Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection

arXiv:2512. 22179v3 Announce Type: replace Abstract: Detecting previously unseen attacks remains a major challenge for machine learning-based intrusion detection systems.

By Rajeeb Thapa Chhetri, Saurab Thapa, Avinash Kumar, Zhixiong Chen

llms diffusion benchmarks

arXiv Machine Learning

Aug 3

GEMSS: A Variational Method for Discovering Multiple Sparse Solutions in Classification and Regression Problems

arXiv:2602. 08913v3 Announce Type: replace Abstract: In underdetermined regression and classification problems, multiple feature subsets often yield equivalent predictive performance.

By Kate\v{r}ina Henclov\'a, V\'aclav \v{S}m\'idl

multimodal benchmarks

arXiv Machine Learning

Aug 3

LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning

arXiv:2603. 13319v2 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising paradigm for parallel token generation, with block-wise variants garnering significant research interest.

By Yanzhe Hu, Yijie Jin, Pengfei Liu, Kai Yu, Zhijie Deng

llms diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Aug 3

POSSE-kNN: Pathwise Out-of-Bag Selected Subspace Ensembles for Binary Classification

arXiv:2211. 11278v3 Announce Type: replace-cross Abstract: Nearest neighbour classification is attractive for tabular data, but its performance can deteriorate when a fixed query centred neighbourhood does not follow the local class geometry.

By Zardad Khan, Amjad Ali, Najd Adeed, Saeed Aldahmani

benchmarks

arXiv AI

Aug 3

Library Reachability in LSR-Synth: How Anti-Memorization Design Changes the Measurement of Symbolic Discovery

arXiv:2607. 28684v1 Announce Type: new Abstract: Existing benchmarks for scientific equation discovery are largely composed of well-known equations available in the public domain, making it difficult to determine whether a model is discovering laws from data or merely recalling answers from its training corpus.

By Zhan'ao Yao, Liang Yin, Zhihao Gao, Boxuan Zhang, Xiaoyu Wu, Linjing Li, Rongyan Wang, Tingwei Chen, Youwei Wang, Xiaolin Zhao, Jiahui Shi, Jianjun Liu

llms benchmarks

arXiv AI

Aug 3

Multi-Agent Planning with Spatio-Temporal and Topological Constraints using STL-GO

arXiv:2607. 28679v1 Announce Type: new Abstract: Multi-agent planning problems arise in a variety of engineering applications, such as multi-robot wildfire fighting and unmanned aerial inspection in factories.

By Sheryl Paul, Vidisha Kudalkar, Anand Balakrishnan, Lars Lindemann, Alberto Speranzon, Jyotirmoy V. Deshmukh

agents robotics benchmarks

arXiv AI

Aug 3

OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systems

arXiv:2607. 28629v1 Announce Type: new Abstract: The rapid transition from reactive large language models (LLMs) to persistent, action-capable systems has exposed critical gaps in the architectural understanding of Agentic AI, particularly in separating inference, orchestration, and execution layers for autonomous AI agents.

By Konstantinos I. Roumeliotis, Ranjan Sapkota

llms agents benchmarks safety

arXiv AI

Aug 3

AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

arXiv:2607. 29626v1 Announce Type: new Abstract: As LLMs evolve from code completion systems into autonomous scientific agents, evaluating their ability to conduct experiments has become increasingly important.

By Tianyu Huai, Tingshuo Fan, Xinchi Chen, Yining Zheng, Yuxin Wang, Shuang Chen, Jie Zhou, Xuanjing Huang

llms agents benchmarks

arXiv AI

Aug 3

COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention

arXiv:2607. 28649v1 Announce Type: cross Abstract: COSI-Lab presents a multimodal, multi-sensor dataset of an interdisciplinary scientific workshop containing 32 academics at an international conference.

By Zonghuan Li, Litian Li, Arthur Mercier, Gara Dorta, Balint Dioszegi, Jose Morales-Vargas, Chenxu Hao, Ivan Kondyurin, Vanessa Begemann, Nale Lehmann-Willenbrock, Bernd Dudzik, Saunaq Chakrabarty, Sotiris Vacanas, Laura Cabrera-Quir\'os, Anne L. J. ter Wal, Vitaliy Popov, Jorge Castro-God\'inez, Chirag Raman, Stephanie Tan, Hayley Hung

multimodal benchmarks safety

arXiv AI

Aug 3

WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization

arXiv:2607. 28699v1 Announce Type: cross Abstract: KV-cache quantization is validated today by offline benchmark averages; a deployed system cannot tell whether compression is damaging the request it is serving right now.

By Fanzhe Wei, Li Liu

efficiency benchmarks

Get the daily briefing