Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,504 stories · RSS feed

arXiv Machine Learning
Jun 29

Aurora: A Leverage-Aware Spectral Optimizer

arXiv:2606. 27715v1 Announce Type: new Abstract: We show that for tall matrix parameters, like projection matrices in the MLP layers, the Muon update can have row norms that are arbitrarily non-uniform.

By Alec Dewulf, Dhruv Pai, Li Yang, Ashley Zhang, Ben Keigwin
arXiv AI
Jun 29

DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection

arXiv:2606. 27499v1 Announce Type: cross Abstract: Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when an agent genuinely needs to remember what it saw rather than what it could write down.

By Yujin Tang, Chenming Shang, Ruize Xu, Nikhil Singh
arXiv AI
Jun 29

Mind the Gap: Quantifying the Domain Gap in Cross-Sensor Diffusion Super-Resolution

arXiv:2606. 28039v1 Announce Type: cross Abstract: Demand for high-resolution satellite imagery has increased interest in super-resolution (SR) to bridge the spatial resolution gap between freely available missions such as Sentinel-2 and commercial systems like PlanetScope.

By Dawid Kope\'c, Katarzyna Jab{\l}o\'nska, Wojciech Koz{\l}owski, Maciej Zi\k{e}ba
arXiv AI
Jun 29

HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration

arXiv:2606. 28215v1 Announce Type: cross Abstract: Extracting dynamic 4D object interactions from massive, in-the-wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs.

By Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao, Su Linxiang, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li