Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,504 stories · RSS feed

arXiv AI
Jun 30

A Task-Driven and Quality-Assured Agent Framework for SAR Data Generation

arXiv:2606. 28896v1 Announce Type: cross Abstract: Synthetic aperture radar (SAR) data augmentation is important for improving the generalization of data-driven SAR interpretation models, yet practical augmentation workflows are often hindered by heterogeneous dataset formats, task-dependent metadata requirements, diverse generation methods, and weak validation of generated samples.

By Xuanting Wu, Fan Zhanga, Fei Ma, Ling Guan, Guochun Ma, Yongsheng Zhou
arXiv Machine Learning
Jun 30

Spatial Deconfounder: Interference-Aware Deconfounding for Spatial Causal Inference

arXiv:2510. 08762v2 Announce Type: replace Abstract: Causal inference in spatial domains faces two intertwined challenges: (1) unmeasured spatial factors, such as weather, air pollution, or mobility, that confound treatment and outcome, and (2) interference from nearby treatments that violate standard no-interference assumptions.

By Ayush Khot, Miruna Oprescu, Maresa Schr\"oder, Ai Kagawa, Xihaier Luo
arXiv Machine Learning
Jun 30

fev-bench: A Realistic Benchmark for Time Series Forecasting

arXiv:2509. 26468v3 Announce Type: replace Abstract: Benchmark quality is critical for meaningful evaluation and sustained progress in time series forecasting, particularly with the rise of pretrained models.

By Oleksandr Shchur, Abdul Fatir Ansari, Caner Turkmen, Lorenzo Stella, Nick Erickson, Pablo Guerron, Michael Bohlke-Schneider, Yuyang Wang
arXiv AI
Jun 30

XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation

arXiv:2412. 15529v4 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output is not only contextually relevant but also accurate and current.

By Qili Zhang, Qianren Mao, Yangyifei Luo, Yashuo Luo, Hanwen Hao, Zhilong Cao, Weifeng Jiang, Zhijun Chen, Junnan Liu, Feng Yan, Xiaolong Wang, Jinlong Zhang, Zhenting Huang, Zhixing Tan, Jie Sun, Bo Li, Jianxin Li, Philip S. Yu