Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv Computer Vision
21h ago

WAM-Sim0.1: World Anchored Memory Meets Online RL for Consistent and Realistic Driving Simulation

WAM‑Sim0.1 is a driving world model that uses a persistent hierarchical memory to anchor context across pose, scene, and instance tokens, addressing structural, identity, and quality drift in autoregressive simulations. It incorporates online flow‑matching reinforcement learning to refine self‑generated rollouts, guided by visual rewards and a vision‑language physical judge, thereby improving visual sharpness and physical plausibility. On the nuScenes benchmark, WAM‑Sim achieves high generation fidelity (FID = 3.47, FVD = 25.72) and strong downstream perception metrics (24.16 mAP, 71.23 road mIoU), while maintaining scene similarity and identity coherence over minute‑scale rollouts.

By Jiahao Cui, Jiaye Li, Wenhao Zhang, Yaoyi Li, Shan Luan, Quanhui Tang, Jianhua Han, Lin Cheng, Hang Xu, Siyu Zhu
arXiv Computer Vision
21h ago

MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception

MITO is the first millimeter‑wave dataset featuring diverse everyday objects, collected with a UR5 robotic arm equipped with two mmWave radars and an RGB‑D camera. It includes over 24 million mmWave frames, 550 high‑resolution mmWave images in both line‑of‑sight and non‑line‑of‑sight conditions, along with RGB‑D images, segmentation masks, and raw signals for 76 objects. An open‑source simulator is provided to generate synthetic mmWave images from any 3D triangle mesh, and benchmarks for NLOS segmentation and classification are established to showcase the dataset’s utility.

By Tara Boroushaki, Laura Dodds, Cusuh Ham, Fadel Adib