arXiv AI
OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 1
RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction
RoboPhys-3D is a 3D‑grounded embodied world model benchmark built on RoboTwin 2.0, featuring 50 manipulation tasks, 5,000 episodes, and 25,000 multi‑view ground‑truth videos. It evaluates video world models by processing both generated and ground‑truth videos through the same 3D reconstruction pipeline, allowing the separation of reconstruction‑induced from generation‑induced errors. The benchmark defines 50 metrics across four sub‑dimensions—pixel fidelity, 3D geometry consistency, state understanding, and task completeness—and introduces the Average Full Score and RoboPhyscore for holistic assessment, with RoboPhyscore showing strong correlation with human judgments.
arXiv AI
Jun 30
RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis
arXiv:2606. 28385v1 Announce Type: cross Abstract: Recent advances in robot world models enable synthetic video generation for embodied prediction and planning.
arXiv AI
Sep 24
EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields
arXiv:2605.06192v2 Announce Type: replace-cross Abstract: Pretrained video diffusion models provide powerful spatiotemporal generative priors, making them a natural foundation for robotic world model...
arXiv Computer Vision
Sep 10
VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models
arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...
arXiv Computer Vision
Sep 21
RobotEQ-Video: A Video-Centric Benchmark for Social Proactive Intelligence with World-State Taxonomy
RobotEQ-Video is a new video-centric benchmark designed to advance Social Proactive Intelligence (SPI) by moving beyond static image analysis. It introduces a hierarchical world-state taxonomy with 6 domains, 20 dimensions, 142 level‑1 attributes, and 816 level‑2 attributes, and includes over 2,000 videos annotated with 100,000+ human labels and 16,000+ behavior‑properness tags. Evaluation shows existing systems underperform humans, highlighting the need for richer video data and comprehensive scenario coverage in SPI research.
arXiv Computer Vision
Oct 15
WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
arXiv:2610.19063v1 Announce Type: new Abstract: As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requi...