Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv Machine Learning

Jul 14

Inter-Stop Energy Prediction and Causal Driver Quantification for Dual-Source Trolleybuses via a Time-Aware Tabular Deep Learning Architecture

arXiv:2607. 11349v1 Announce Type: cross Abstract: Dual-source trolleybuses alternate between overhead catenary supply and on-board battery operation, creating energy-use patterns driven by route attributes, high-frequency trajectories, and hourly weather.

By Wentao Zeng (School of Management, Foshan University, Foshan, China a School of Management, Foshan University, Foshan, China, School of Mechanical and Electrical Engineering and Automation, Foshan University, Foshan, China), Zijian Huang (School of Artificial Intelligence, South China Normal University, Guangzhou, China), Yiming Bie (School of Transportation, Jilin University, Changchun, China), Jiabin Wu (School of Management, Foshan University, Foshan, China a School of Management, Foshan University, Foshan, China), Jun Gong (Department of Civil Engineering, The University of Hong Kong, Hong Kong, China)

benchmarks

arXiv Machine Learning

Jul 14

Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion

arXiv:2506. 22036v2 Announce Type: replace Abstract: With the increasing multimodal knowledge privatization requirements, multimodal knowledge graphs in different institutes are usually decentralized, lacking of effective collaboration system with both stronger reasoning ability and transmission safety guarantees.

By Ying Zhang, Yu Zhao, Xuhui Sui, Baohang Zhou, Xiangrui Cai, Li Shen, Xiaojie Yuan, Dacheng Tao

rag diffusion efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 14

A computational model of infant sensorimotor exploration in the mobile paradigm

arXiv:2504. 17939v2 Announce Type: replace-cross Abstract: We present a computational model of the mechanisms that may determine infant behavior in the "mobile paradigm".

By Josua Spisak, Sergiu Tcaci Popescu, Stefan Wermter, Matej Hoffmann, J. Kevin O'Regan

llms benchmarks

arXiv AI

Jul 14

Uncertainty Quantification for EO Regression Tasks: Building Height, Tree Canopy Height and Above-ground Biomass Estimation

arXiv:2607. 11412v1 Announce Type: cross Abstract: Earth Observation regression tasks such as building height, canopy height, and above-ground biomass estimation underpin critical applications in urban planning, forest monitoring, and climate policy, where both accuracy and reliability are critical.

By Ritu Yadav, Andrea Nascetti, Yifang Ban

benchmarks

arXiv AI

Jul 14

Longitudinal Multi-View Breast Cancer Risk Prediction

arXiv:2607. 11343v1 Announce Type: cross Abstract: Accurate breast cancer risk prediction from screening mammography is critical for enabling personalized screening intervals and early detection.

By Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstr{\o}m, Elisabeth Wetzer, Stine Hansen, Robert Jenssen, Michael Kampffmeyer

benchmarks safety

arXiv AI

Jul 14

Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models

arXiv:2603. 22042v3 Announce Type: replace-cross Abstract: While Vision-Language Models (VLMs) have achieved remarkable performance, their Euclidean embeddings remain limited in capturing hierarchical relationships such as part-to-whole or parent-child structures, and often face challenges in multi-object compositional scenarios.

By Hayeon Kim, Ji Ha Jang, Junghun James Kim, Se Young Chun

llms rag multimodal benchmarks safety

arXiv AI

Jul 14

The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions

arXiv:2607. 10911v1 Announce Type: cross Abstract: In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications.

By Filip Klubicka, Vasudevan Nedumpozhimana, Sneha Rautmare, Bora Caglayan, Mingxue Wang, John D. Kelleher

llms fine-tuning benchmarks data-engineering

arXiv AI

Jul 14

Enhancing LLMs through human feedback: a journey towards self-improvement

arXiv:2607. 11267v1 Announce Type: cross Abstract: In the rapidly evolving landscape of information retrieval systems, the ability to adapt and improve through user feedback is paramount.

By Tatiana Pelc, Gila Kamhi, Asaf Avrahamy, Adi Fledel-Alon

llms rag agents benchmarks

arXiv AI

Jul 14

Graph-Based Bayesian Optimization for Quantum Circuit Architecture Search with Uncertainty Calibrated Surrogates

arXiv:2512. 09586v2 Announce Type: replace-cross Abstract: Quantum circuit design is a key bottleneck for practical quantum machine learning on complex, real-world data.

By Prashant Kumar Choudhary, Nouhaila Innan, Muhammad Shafique, Rajeev Singh

rag benchmarks

arXiv AI

Jul 14

SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data

arXiv:2602. 14687v2 Announce Type: replace-cross Abstract: Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations.

By David Chanin, Adri\`a Garriga-Alonso

llms benchmarks

arXiv Machine Learning

Jul 14

A multi-scale feature enhanced graph neural network for fluid dynamics prediction in complex geometries

arXiv:2607. 11672v1 Announce Type: new Abstract: Industrial design in fields such as vehicle and aerospace engineering often relies on large-scale numerical simulations to evaluate fluid dynamics performance, which can incur substantial computational costs.

By Li Xiao, Tianyu Li, Yiye Zou, Mingjie Zhang, Xiaogangd Deng

benchmarks

arXiv Machine Learning

Jul 14

A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol

arXiv:2607. 11873v1 Announce Type: cross Abstract: Institutions collect far more open-ended teaching-evaluation feedback than they read.

By Esteban U. Vega Barajas

llms rag benchmarks

arXiv AI

Jul 14

FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion

arXiv:2607. 10140v1 Announce Type: cross Abstract: Existing optical flow methods broadly follow two paradigms: iterative optimization and diffusion-based estimation.

By Yuang Meng, Chenyang Wu, Xianshun Liu, Chun-Le Guo, Zichen Liang, Lina Lei, Jie Liang, Hui Zeng, Chongyi Li, Lei Zhang

diffusion computer-vision benchmarks

arXiv AI

Jul 14

Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series

arXiv:2607. 10740v1 Announce Type: cross Abstract: The analysis of Multivariate Time Series (MTS) plays an important role in a lot of real-world practical applications, but it still remains some challenging problem about capturing multi-granularity structural patterns and suppressing noise appropriately.

By HaoChong Fu, Jian Xu

rag benchmarks

arXiv AI

Jul 14

Metacognition in LLMs: Foundations, Progress, and Opportunities

arXiv:2607. 11881v1 Announce Type: cross Abstract: Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more.

By Gabrielle Kaili-May Liu, Areeb Gani, Jacqueline Lu, Jordan Thomas, Mark Steyvers, Arman Cohan

llms nlp benchmarks

arXiv AI

Jul 14

Algorithm Selection with Zero Domain Knowledge via Text Embeddings

arXiv:2604. 19753v2 Announce Type: replace Abstract: We propose a feature-free approach to algorithm selection: instead of hand-crafted instance features, we use pretrained text embeddings.

By Stefan Szeider

rag benchmarks

arXiv AI

Jul 14

The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy

arXiv:2607. 11175v1 Announce Type: new Abstract: The growing ability of large language models and vision language models to jointly interpret and reason over images and text is reshaping medical agents, moving them from task specific predictors toward autonomous systems that perceive, reason, plan, remember, and act in clinical environments.

By Chunzheng Zhu, Lei Tian, Bohan Tan, Ziqi Zhou, Yuxuan Sun, Yijun Wang, Chengchao Lv, Yilin Wen, Yijun He, Jinghao Lin, Yihang Chen, Cheewei Tan, Qianshan Wei, Lei Zhao, Bin Pu, Kenli Li, Yuan Xue, Jianxin Lin

llms agents benchmarks safety

arXiv AI

Jul 14

MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models

arXiv:2603. 09714v2 Announce Type: replace-cross Abstract: While multi-audio understanding is critical for large audio-language models (LALMs), it remains underexplored.

By Chih-Kai Yang, Yun-Shao Tsai, Yu-Kai Guo, Ping-Le Tsai, Yen-Ting Piao, Hung-Wei Chen, Ting-Lin Hsiao, Yun-Man Hsu, Ke-Han Lu, Hung-yi Lee

llms benchmarks

arXiv Machine Learning

Jul 14

Conservation Laws for Diffusion Models

arXiv:2607. 10067v1 Announce Type: new Abstract: While autoregressive models optimize the exact data likelihood via the chain rule, diffusion models are typically trained with denoising objectives.

By Ziv Aharoni, Henry D. Pfister

diffusion benchmarks

arXiv Machine Learning

Jul 14

Neural Discovery of Memory and Nonlocal Kernels in Integro-Differential Equations with Constrained Kolmogorov--Arnold Networks

arXiv:2607. 11110v1 Announce Type: new Abstract: Discovering the memory or nonlocal kernel governing an integro-differential equation (IDE) from sparse and noisy observations is an ill-posed inverse problem.

By Aruzhan Tleubek, Salah A Faroughi

diffusion benchmarks

Get the daily briefing