Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

15,528 stories · RSS feed

arXiv Machine Learning
Aug 4

AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.

By Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana, Gopi Krishna Nedanuri
arXiv Machine Learning
Aug 4

MedSAM2-Anatomy: Training-Free Inference-Time Optimization for Musculoskeletal Segmentation

arXiv:2608. 00195v1 Announce Type: cross Abstract: High-resolution 3D segmentation of hip and shoulder anatomy from CT and MRI is essential for surgical planning, yet frozen segmentation models often fail under domain shift.

By John Garcia Henao, Nicholas B\"unger, Benedikt Herzog, Cindy Guerrero Toro, Benjamin Vella, Matthias Biner, Rico Br\"utsch, Carmen Castroviejo Fernandez, Felix \"Ottl, Norman Juchler, Armando Hoch, Bettina Hochreiter, Sven Hirsch, Sebastiano Caprara
arXiv Machine Learning
Aug 4

Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams

arXiv:2608. 00012v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster emergency response remains insufficiently evaluated.

By Fengxiang Wang, Qiuyang Yu, Yueying Li, Mingshuo Chen, Chengchi Fei, Kaiyi Xu, Lixin Gu, Wangxu Wei, Junchao Gong, Lipeng Ma, Jiong Wang, Fenghua Ling, Wenlong Zhang, Xue Yang, Wenjing Yang, Ben Fei, Long Lan