Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

14,014 stories · RSS feed

arXiv Machine Learning
Aug 11

TRAPS: Treatment-Assignment Prediction via Pathway-informed Stratification

arXiv:2606. 09898v2 Announce Type: replace Abstract: Cancer treatment involves decisions across multiple clinical outcomes, yet pathway-informed deep learning models are typically evaluated in isolation, making their relative benefits unclear.

By Sujoy Banik, Sayantan Chakraborty, Boishakhi Das Toma, Zainab Ghafoor, Ushashi Bhattacharjee, Koushik Howlader, Tirtho Roy
arXiv Machine Learning
Aug 11

V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control

arXiv:2608. 07870v1 Announce Type: new Abstract: Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly.

By Donghu Kim, Youngdo Lee, Hojoon Lee, Johan Obando-Ceron, Byungkun Lee, Aaron Courville, Pablo Samuel Castro, Jaegul Choo, Clare Lyle
arXiv Machine Learning
Aug 11

Opportunity Is Not Realizability: Selection-Valid Diagnostics for Multi-LLM Routing

arXiv:2608. 08265v1 Announce Type: new Abstract: Oracle routing measures how much a pool of language models could gain from per-query selection, but the diagnostic has two flaws: testing against a best fixed model selected on the same examples invalidates paired inference, and a full-information oracle sees outcomes no deployable router observes.

By Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus Swaqeeb
arXiv Machine Learning
Aug 11

Classical $\mathrm{SU}(2)$ Models Match or Exceed Shallow Variational Quantum Circuits on Vision Benchmarks

arXiv:2608. 07822v1 Announce Type: cross Abstract: Quaternion-valued neural networks and variational quantum circuits (VQCs) both derive local transformations from $\mathrm{SU}(2)$ geometry, yet their performance on classical supervised learning remains poorly understood.

By Christopher Fulton, Irene Tsapara, Lawrence Fulton
arXiv Machine Learning
Aug 11

PreGress: Ranking-Native Pre-training and Prompting for Graph Node Ranking

arXiv:2608. 09016v1 Announce Type: cross Abstract: Node ranking is a fundamental problem in graph information retrieval, measuring the relative importance of nodes and supporting a wide range of applications such as influence analysis, recommendation, and graph-based retrieval augmented generation.

By Lujie Ban, Jiasheng shi, Yingli Zhou, Kaiwen Xue, Daiyin Wang, Xubin Li, Shuanghua Li, Chenhao Ma
arXiv Machine Learning
Aug 11

Disentangling Co-Occurring Retinal Pathologies with Saliency-Guided Sparse Expert Routing

arXiv:2608. 09752v1 Announce Type: cross Abstract: Retinal fundus images frequently exhibit multiple co-occurring pathologies, yet standard deep learning classifiers apply static, identical computation to every image regardless of the underlying disease distribution.

By Nagur Shareef Shaik, Jeongwoo Park, Yeong-Jin Kim, Jaeuk Jung, Hyunjung Oh, Dong Hye Ye