Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,065 stories · RSS feed

arXiv Machine Learning

Jul 22

Subject-Conditioned Glucose Forecasting in Type-1 Diabetes

arXiv:2607. 19006v1 Announce Type: new Abstract: Accurate forecasting of blood glucose concentration is key in the management of Type 1 Diabetes, facilitating early detection of adverse glycemic events and supporting timely therapeutic interventions.

By Giorgia Rigamonti, Mirko Paolo Barbato, Davide Marelli, Paolo Napoletano

multimodal benchmarks

arXiv AI

Jul 22

MambaLSTM: A Spatio-Temporal Framework for Enhanced Traffic Accident Risk Prediction

arXiv:2607. 18353v1 Announce Type: cross Abstract: In traffic accident risk prediction, most studies overlook the extra noise that could be incorporated when fusing temporal features into spatial features, and some models struggle to capture global correlations among spatial regions.

By Zhen Yu, Yachao Yuan, Zixiang Peng, Muting Li, Thar Baker

rag benchmarks

arXiv Machine Learning

Jul 22

Budgeted Indirect Adversarial Attack on Graph-Based Anomaly Detection in Sensor Networks

arXiv:2509. 17987v2 Announce Type: replace Abstract: Graph Neural Networks (GNNs) have emerged as powerful models for anomaly detection in sensor networks, particularly when analyzing multivariate time series.

By Sanju Xaviar, Omid Ardakanian

efficiency benchmarks safety

arXiv AI

Jul 22

Do AI-Native Biotechs Need Departments? Benchmarking Company World Models for AI-Driven Drug Development

arXiv:2607. 18696v1 Announce Type: new Abstract: AI-native biotechnology companies are often designed by copying human biotech org charts into agent roles.

By Yinan Wang

agents benchmarks

arXiv AI

Jul 22

Semantic Cooperative Games for Contribution Attribution in LLM-Based Multi-Agent Systems

arXiv:2607. 18255v1 Announce Type: new Abstract: Contribution attribution has become a central problem in LLM-based multi-agent systems, where final outputs are produced through multiple agents, message exchanges, and ordered workflow dependencies.

By Pengyi Jiang, Xiaoguang Zhu, Quanyan Zhu

llms agents benchmarks

arXiv AI

Jul 22

BioSecBench-Surveillance: A Verifiable Benchmark for AI Agents in Pathogen Genomic Surveillance

arXiv:2607. 19262v1 Announce Type: new Abstract: As pathogen genomic surveillance scales, the bottleneck is shifting from data generation to analysis.

By Harmon Bhasin, Kevin Flyangolts, Dianzhuo Wang, Evan Seeyave, Arjun Banerjee, Amanda Darling, Joshua Stallings, David Stern, Shawn Higdon, Claire Duvallet, Bryan Tegomoh, Kenny Workman

llms agents benchmarks

arXiv AI

Jul 22

Bridging the Last Mile of Circuit Design: PostEDA-Bench, a Hierarchical Benchmark for PPA Convergence and DRC Fixing

arXiv:2605. 06936v3 Announce Type: replace-cross Abstract: LLM-based agents are increasingly applied to the "last mile" of Electronic Design Automation (EDA): repairing residual sign-off Design Rule Check (DRC) violations and converging Power-Performance-Area (PPA) targets after tool runs.

By Pengju Liu, Nuo Xu, Jinwei Tang, Yu Cao, Caiwen Ding

llms agents benchmarks

arXiv AI

Jul 22

CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

arXiv:2602. 08686v3 Announce Type: replace-cross Abstract: Prefill-only KV compression freezes a token subset at the end of prefill and decodes from it without further eviction.

By Ning Yang, Chengzhi Wang, Yibo Liu, Baoliang Tian, Haijun Zhang

benchmarks

arXiv AI

Jul 22

Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation

arXiv:2602. 15875v2 Announce Type: replace-cross Abstract: Current Visual-Language Navigation (VLN) methodologies face a trade-off between semantic understanding and control precision.

By Zhenxing Xu, Yihong Lu, Weidong Bao, Zhengqiu Zhu, Jingxuan Zhou, Zhichuang Wang, Ji Wang, Lihua Liu, Wei He

llms multimodal benchmarks

arXiv Machine Learning

Jul 22

Uncertainty Quantification for AI-Driven Crash Simulation Surrogates: A Comparative Study of Monte Carlo Dropout and Deep Ensemble on Open-Source Bumper Beam Benchmark

arXiv:2607. 18294v1 Announce Type: new Abstract: Machine learning surrogate models are increasingly being explored in engineering product development to augment simulation-driven design, offering near-instantaneous predictions that complement computationally expensive high-fidelity analyses.

By Sudeep Chavare

benchmarks safety

arXiv AI

Jul 22

OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining

arXiv:2607. 18927v1 Announce Type: new Abstract: We present OntoBook, a method that converts medical ontology structure into pretraining signal for encoder language models.

By Rian Touchent (ALMAnaCH), \'Eric de la Clergerie (ALMAnaCH)

llms benchmarks safety

arXiv Machine Learning

Jul 22

H$^2$SD: Hybrid Hindsight Self-Distillation

arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.

By Qiye Cai, Yichuan Ma, Linyang Li, Peiji Li, Yongkang Chen, Qipeng Guo, Yicheng Zou, Tao Gui, Xiaocheng Feng, Bing Qin

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jul 22

DWM: Separating World Effects from Actions in Latent World Models

arXiv:2607. 18715v1 Announce Type: new Abstract: Latent world models underpin much of modern model-based control, yet current action-conditioned formulations supervise the next-latent transition with a single, undifferentiated target, forcing a monolithic learning signal to absorb every source of state change.

By Yi-Ge Zhang, Tianqi Du, Qi Zhang, Yisen Wang

agents benchmarks

arXiv AI

Jul 22

Physics-Guided Masked Multi-Task Network for Edge-Friendly Battery Health Diagnostics from Sto-chastically Fragmented Charging Profiles

arXiv:2607. 18330v1 Announce Type: cross Abstract: The deployment of reliable lithium-ion battery management systems is crucial for accelerating electrification, yet the joint prognosis of State of Health (SOH) and Remaining Useful Life (RUL) remains severely hindered by task heteroscedasticity.

By Shuhao Chen, Tianyu Shi, Chengyi Tu

llms rag benchmarks

arXiv AI

Jul 22

Towards an Automated Test of LLM Security Knowledge

arXiv:2607. 18496v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for a range of software, hardware and human-centered security tasks.

By Shufan Chai, Liangliang Sun, Jessica Staddon

llms benchmarks

arXiv AI

Jul 22

Dynamic Loss Balancing for Joint SOH and RUL Prediction of Lithium-Ion Batteries via a Rotary SOH-Injected Prior Battery Transformer

arXiv:2607. 18329v1 Announce Type: cross Abstract: The deployment of reliable lithium-ion battery management systems is crucial for accelerating electrification, yet the joint prognosis of State of Health (SOH) and Remaining Useful Life (RUL) remains severely hindered by task heteroscedasticity.

By Shuhao Chen, Tianyu Shi, Yiwen Huang, Chengyi Tu

llms rag benchmarks

arXiv AI

Jul 22

Parameter-Efficient Continual Fine-Tuning: A Survey

arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.

By Eric Nuertey Coleman, Luigi Quarantiello, Ziyue Liu, Qinwen Yang, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco

fine-tuning benchmarks

arXiv Machine Learning

Jul 22

One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models

arXiv:2607. 19128v1 Announce Type: new Abstract: Vision-language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored.

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan

llms rag multimodal benchmarks

Hugging Face Trending Papers

Jul 22

ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment

Automatic pain assessment from facial video remains challenging due to the spatial heterogeneity of pain-related facial cues. This study proposes ReFace, a spatial reorganization pipeline that divides facial input into four spatial quadrants before tokenization, rather than processing the entire face as a single region.

nlp benchmarks

Hugging Face Trending Papers

Jul 22

PhenSPINE: A Standardized Benchmark for Spine Pathology Diagnosis

The accurate diagnosis of spinal pathologies depends heavily on radiological interpretation, yet automated systems are hindered by the lack of diverse, high-quality benchmarks. In this study, we present PhenSPINE, a Magnetic Resonance Imaging dataset comprising 16,813 images from 250 patients, curated to facilitate advanced deep learning research.

benchmarks

Get the daily briefing