Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,633 stories · RSS feed

arXiv Machine Learning

Jul 21

ECG-LLM: Foundation Model for ECG-Based Cardiac Reasoning

arXiv:2607. 16323v1 Announce Type: cross Abstract: Electrocardiography (ECG) is an inexpensive, standard-of-care test for cardiac symptoms, but front-line triage often lacks immediate access to definitive imaging such as echocardiography (ECHO) or cardiac magnetic resonance (CMR).

By Alexander Selivanov, Friederike Jungmann, Jan Kehrer, Karl-Ludwig Laugwitz, Eimo Martens, Daniel Rueckert

llms multimodal benchmarks

arXiv AI

Jul 21

Privacy-Aware Synthetic Video Benchmarking and Relational Evaluation for Worker-Under-Suspended-Load Detection

arXiv:2607. 16351v1 Announce Type: cross Abstract: Publicly shareable construction-video benchmarks remain scarce, especially for safety-critical hazards that are rare, dangerous to stage, and difficult to release.

By Anshu Singh, Alejandro Seif

diffusion computer-vision benchmarks safety

arXiv AI

Jul 21

The Shared Discovery Paradox: How a One-Answer Rule Turns Better Information into Worse Search

arXiv:2607. 18045v1 Announce Type: new Abstract: Organizations often pool dispersed information into one ranking and then allow many agents to act on that shared view.

By Yohei Nakajima

agents benchmarks

arXiv Machine Learning

Jul 21

Spectral-Morphological Attention U-Net: An Efficient Network for Active Wildfire Detection

arXiv:2607. 16472v1 Announce Type: cross Abstract: Over the past decades, the frequency of global wildfires has been increasing steadily.

By Yugong Zeng, Jonathan Wu

computer-vision benchmarks

arXiv AI

Jul 21

What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning

arXiv:2607. 16938v1 Announce Type: cross Abstract: End-to-end autonomous driving models are now able to navigate complex road scenarios, mapping raw sensor observations directly to observed paths for open-loop evaluation and often effective driving in closed-loop evaluation.

By Kalpana Panda, Wesley Maia, Vinti Agarwal, Ross Greer

agents robotics multimodal benchmarks safety

arXiv AI

Jul 21

Generative Ontology Induction: Domain-Agnostic Schema Discovery from Document Corpora Using Large Language Models

arXiv:2607. 16201v1 Announce Type: new Abstract: Ontology engineering remains a critical bottleneck in knowledge-intensive AI systems.

By Sergei Sergienko

llms benchmarks

arXiv AI

Jul 21

Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making

arXiv:2607. 17038v1 Announce Type: new Abstract: This paper addresses key technical challenges in current large language model (LLM) agent applications, including long-horizon planning, sparse reward attribution, and dynamic environmental interaction, by designing and optimizing an intelligent agent workflow.

By Amez Amanj Ali, Kuo-Kun Tseng

llms agents reinforcement-learning robotics multimodal benchmarks safety

arXiv AI

Jul 21

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.

By Longteng Zhang, Sen Wu, Shuai Hou, Zhengyu Qing, Zhuo Zheng, Danning Ke, Qihong Lin, Qiang Wang, Shaohuai Shi, Xiaowen Chu

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 21

CDIS: Cross-Dimensional Class-Agnostic 3D Instance Segmentation via 2D Mask Tracking and 3D-2D Projection Merging

arXiv:2607. 17778v1 Announce Type: cross Abstract: Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments, enabling perception of previously unseen objects for reliable manipulation and navigation.

By Juno Kim, Hye-Jung Yoon, Yesol Park, Byoung-Tak Zhang

computer-vision robotics benchmarks

arXiv AI

Jul 21

PEARL: Auditable Repair for Scientific Reasoning Graph Extraction

arXiv:2607. 17917v1 Announce Type: new Abstract: Scientific Reasoning Graph Extraction (SRGE) aims to recover explicit links among observations, evidence, intermediate claims, and paper-level conclusions.

By Bohan Su, Pengze Li, Yuchen Lu, Xi Chen

llms agents benchmarks

arXiv Machine Learning

Jul 21

Regret Optimality of Sample Average Approximation for Data-Driven Newsvendor Problems: A General Optimization Perspective

arXiv:2407. 04900v2 Announce Type: replace Abstract: Numerous existing studies have examined the performance of Sample Average Approximation (SAA) in the fundamental newsvendor problem.

By Jiameng Lyu, Shilin Yuan, Bingkun Zhou, Yuan Zhou

benchmarks

arXiv Machine Learning

Jul 21

Robust Losses from Univariate Base Functions for Noisy-Label Learning

arXiv:2607. 16768v1 Announce Type: new Abstract: Learning with noisy labels is a fundamental problem in training reliable deep neural networks.

By Peng Hu, Jianwei Ma

benchmarks

arXiv AI

Jul 21

Autonomous Discovery of Wireless Communications Algorithms

arXiv:2607. 17762v1 Announce Type: cross Abstract: Large language model (LLM)-driven evolutionary search is an emerging algorithm-discovery paradigm that has already produced novel results in several scientific fields.

By Fay\c{c}al A\"it Aoudia, Jakob Hoydis, Sebastian Cammerer, Gian Marti, Merlin Nimier-David, Nicolas Roussel, Alexander Keller

llms agents benchmarks

arXiv Machine Learning

Jul 21

Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning

arXiv:2607. 18130v1 Announce Type: new Abstract: Most parameter-efficient finetuning (PEFT) methods adapt weights or activations, thus leaving one of the key Transformer components unchanged: residual connections.

By Valentijn Oldenburg, Floris de Kam, Bente Zuijdam, Lieve Eberson, Nicky van Zutphen, Stef de Wildt, Ivo Verhoeven

llms fine-tuning benchmarks

arXiv AI

Jul 21

HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization

arXiv:2607. 17994v1 Announce Type: cross Abstract: Video understanding has become more and more important with the growth of Artificial Intelligence (AI) for video generation.

By Rui Chu, Yingjie Lao

llms diffusion nlp multimodal benchmarks

arXiv Machine Learning

Jul 21

WINO: A Weak-Form Physics Informed Neural Operator for Hyperelasticity on Variable Domains

arXiv:2605. 24651v2 Announce Type: replace-cross Abstract: We propose a Weak-form Physics-Informed Neural Operator (WINO), a data-free framework that combines the efficiency of neural operators with the geometric flexibility of the $\varphi$-finite element method ($\varphi$-FEM).

By Bokai Zhu, Yizheng Wang, Qinghui Zhang, Timon Rabczuk

benchmarks

arXiv AI

Jul 21

Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction

arXiv:2607. 16250v1 Announce Type: cross Abstract: Estrogen Receptor (ER) status is a critical biomarker in breast cancer diagnosis, prognosis, and treatment selection.

By Priyanka Paudel, Madan Baduwal

efficiency benchmarks

arXiv AI

Jul 21

SAGA: Synthetic Agentic Graph Architecture for Temporal Benchmark Generation

arXiv:2607. 17288v1 Announce Type: cross Abstract: High quality temporal graph benchmarks with rich semantics and ground-truth anomaly labels are essential for training graph neural networks, yet remain scarce due to privacy constraints and annotation costs.

By Jiacheng Ding, Xiaofei Zhang

llms rag agents benchmarks safety

arXiv Machine Learning

Jul 21

MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning

arXiv:2607. 16789v1 Announce Type: new Abstract: Real-world perception and decision making are inherently multimodal, integrating complementary signals across modalities.

By Sana Tonekaboni, Viktoria Schuster, Caroline Uhler

fine-tuning multimodal benchmarks safety

arXiv AI

Jul 21

Diffusion-corrected Autoregressive Fourier Neural Operator for Droplet Evolution Prediction

arXiv:2607. 16238v1 Announce Type: cross Abstract: Predicting droplet evolution in material jetting, or Inkjet Printing (IJP), is essential for maintaining printing quality.

By Jinghao Cao, Minsung Kang, Hongyue Sun, Chi Zhou, Jihoon Chung, Xubo Yue, Sanchoy Das, Bo Shen

diffusion benchmarks

Get the daily briefing