Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

14,220 stories · RSS feed

arXiv Machine Learning
Aug 4

AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.

By Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana, Gopi Krishna Nedanuri
arXiv Machine Learning
Aug 4

MedSAM2-Anatomy: Training-Free Inference-Time Optimization for Musculoskeletal Segmentation

arXiv:2608. 00195v1 Announce Type: cross Abstract: High-resolution 3D segmentation of hip and shoulder anatomy from CT and MRI is essential for surgical planning, yet frozen segmentation models often fail under domain shift.

By John Garcia Henao, Nicholas B\"unger, Benedikt Herzog, Cindy Guerrero Toro, Benjamin Vella, Matthias Biner, Rico Br\"utsch, Carmen Castroviejo Fernandez, Felix \"Ottl, Norman Juchler, Armando Hoch, Bettina Hochreiter, Sven Hirsch, Sebastiano Caprara
arXiv Machine Learning
Aug 4

Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams

arXiv:2608. 00012v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster emergency response remains insufficiently evaluated.

By Fengxiang Wang, Qiuyang Yu, Yueying Li, Mingshuo Chen, Chengchi Fei, Kaiyi Xu, Lixin Gu, Wangxu Wei, Junchao Gong, Lipeng Ma, Jiong Wang, Fenghua Ling, Wenlong Zhang, Xue Yang, Wenjing Yang, Ben Fei, Long Lan
arXiv Machine Learning
Aug 4

Diffusion Policy with Behavioral Advantage Correction for Offline Reinforcement Learning

arXiv:2608. 02332v1 Announce Type: new Abstract: In offline reinforcement learning (RL), the distribution shift between behavioral data and the learned policy can lead to erroneous \emph{Q}-value estimation, thereby misguiding the direction of policy optimization.

By Botao Dong, Longyang Huang, Ning Pang, Hongtian Chen
arXiv Machine Learning
Aug 4

A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs

arXiv:2603. 07475v4 Announce Type: replace-cross Abstract: Autoregressive (AR) language models build representations incrementally via left-to-right prediction, while diffusion language models (dLLMs) are trained through full-sequence denoising.

By Raghavv Goel, Risheek Garrepalli, Sudhanshu Agrawal, Chris Lott, Mingu Lee, Fatih Porikli
arXiv Machine Learning
Aug 4

DynamicManip: Enabling Dynamic Manipulation from a Single Static Demonstration

arXiv:2608. 01452v1 Announce Type: cross Abstract: Dynamic manipulation is a critical capability for robots operating in complex and dynamic environments, where robots must interact with objects that are moving or require rapid adjustments.

By Haoran Liao, Pengyue Wang, Shuoyu Chen, Kehan Cheng, Xuhang Chen, Yuhao Lin, Mu Lin, Zhizhao Liang, Xiaoyi Fan, Chengyi Xing, Dan Niu, Yi-Lin Wei, Wei-Shi Zheng