Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

15,528 stories · RSS feed

arXiv Machine Learning
Aug 4

Learned Digital Over-the-Air Computing for Federated Edge Learning

arXiv:2509. 16577v2 Announce Type: replace Abstract: Over-the-air (OTA) aggregation enables federated edge learning (FEEL) by exploiting the superposition property of the wireless channel to merge communication with computation, eliminating the need to schedule and decode devices individually.

By Antonio Tarizzo, Mohammad Kazemi, Deniz G\"und\"uz
arXiv Machine Learning
Aug 4

T-TAMER: Provably Taming Trade-offs in ML Serving

arXiv:2509. 22992v2 Announce Type: replace Abstract: As machine learning models continue to grow in size and complexity, efficient serving faces increasingly broad trade-offs spanning accuracy, latency, resource usage, and other objectives.

By Yuanyuan Yang, Ruimin Zhang, Jamie Morgenstern, Haifeng Xu
arXiv Machine Learning
Aug 4

EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

arXiv:2601. 17883v3 Announce Type: replace Abstract: Electroencephalography (EEG) foundation models (FMs) have recently emerged as a promising paradigm for brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings.

By Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo, Dongrui Wu
arXiv Machine Learning
Aug 4

Retrieval of Coastal Biogeochemical Parameters From Near-Surface Hyperspectral Remote Sensing Reflectance Using Physics-Aware Meta-Learning

arXiv:2605. 05623v3 Announce Type: replace Abstract: Hyperspectral in situ sensing has shown promise in retrieving aquatic biogeochemical (BGC) parameters, such as total suspended solids, dissolved organic carbon, and total chlorophyll-a, for cost-effective monitoring of coastal water quality.

By Yiqing Guo, Nagur R. C. Cherukuru, Eric A. Lehmann, S. L. Kesav Unnithan, Tim J. Malthus, Gemma Kerrisk, Xiubin Qi, Faisal Islam, Tisham Dhar, Mark J. Doubell
arXiv Machine Learning
Aug 4

Local-Time Riemannian Score Matching on the Quantum Pure-State Manifold

arXiv:2605. 03573v4 Announce Type: replace-cross Abstract: Score-based diffusion can be defined intrinsically on the manifold of quantum pure states, $\mathbb{CP}^{d-1}$ with the Fubini--Study metric, but no closed-form transition density is available, so the score must be supervised by a local-time teacher taken from the Euclidean limit of the diffusion in normal coordinates.

By Jian Xu, Wei Chen, Shigui Li, Chao Li, Delu Zeng, John Paisley, Qibin Zhao
arXiv Machine Learning
Aug 4

Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

arXiv:2608. 00623v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains.

By Yinlin Zhu, Di Wu, Yi Zhang, Xunkai Li, Wang Luo, Wei-Jin Huang, Miao Hu, Guocong Quan
arXiv Machine Learning
Aug 4

Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning

arXiv:2608. 00301v1 Announce Type: new Abstract: Error-penalized scoring rules ($+1$ for a correct answer, $-\lambda$ for a wrong one, $0$ for abstaining) are increasingly prescribed against hallucination: a rational agent facing such a rule answers exactly when its correctness probability exceeds Chow's threshold $t^\ast=\lambda/(1+\lambda)$.

By Xujun Che, Yuchen Yuan, Weida Zhao, Chenyang Yu
arXiv Machine Learning
Aug 4

RHEA: Reliability-Harmonized Reconstruction and Assignment for Robust Multimodal-Attributed Graph Clustering

arXiv:2608. 00621v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry heterogeneous attributes such as text and images over a relational structure, have become a fundamental substrate for label-free entity grouping tasks, including community discovery and product segmentation.

By Yinlin Zhu, Di Wu, Ziyu Han, Zekai Chenm, Wang Luo, Miao Hu, Guocong Quan