arXiv AI

What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs

arXiv Computer Vision
Sep 22

Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models

The paper examines latent action models that encode transitions between video frames using algebraic constraints such as additive composition and antisymmetric reversal. It demonstrates that these algebraic consistency conditions do not reliably certify temporal structure, as unconstrained models can achieve similar error reductions and constrained models still outperform unconstrained ones even after temporal pairings are destroyed. The authors find that preserving temporal pairing offers no consistent advantage on downstream tasks and that a direct repair objective yields only marginal improvement, recommending a more rigorous validation protocol.

By Di Wen, Ruodi Zhang, Kailun Yang, Kunyu Peng
arXiv Computer Vision
Aug 28

R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models

R2M-Bench is a benchmark that evaluates revisit memory in interactive video world models by comparing a revisit pair to two control pairs from the same rollout: a gap‑matched non‑revisit pair and a short‑range pair. It introduces MemoryGain (MG) and Normalized Memory Ratio (NMR) to quantify the revisit advantage over generic temporal stability and normalize it by short‑to‑baseline dynamics. Across 300 instances and seven models, NMR correlates with human judgments and reduces the influence of slow‑motion artifacts, with DreamX‑World‑Memo achieving the highest NMR.

By Qiwen Gu, Bingjie Gao, Rui Chen, Geng Li, Jifan Li, Qishuai Wen, Li Niu, Jing Tang, Xiangxiang Chu, Junqiao Zhao
arXiv Computer Vision
Oct 9

Slot3R: Set-Associative Spatial Memory for Streaming 3D Reconstruction

Slot3R introduces a set‑associative spatial memory that allows multiple states to coexist at the same 3D location, addressing the limitation of Point3R where spatial proximity forces merging of distinct observations. By keeping the pretrained Point3R backbone frozen and adding a bounded sparse readout, Slot3R improves point‑cloud accuracy by up to 72% on benchmark datasets and reduces trajectory error across pose benchmarks, while maintaining real‑time performance up to 1000 frames without exhausting memory.

By Xiyuan Zhang, Yanming Yang, Kaiyuan Xu, Ruibo Li, Chi Zhang
arXiv AI
Jun 17

OmniDrive: An LLM-Choreographed Multi-Agent World Model with Unified Latent Co-Compression for Multi-View Driving Video Generation

arXiv:2606. 17536v1 Announce Type: cross Abstract: Generative world models for autonomous driving face two unresolved tensions: heterogeneous control injection, where free-form language, HD-maps, trajectories, and camera poses reside in incompatible representational spaces, and post-hoc cross-view fusion, where per-camera latents fail to encode global 3-D geometry.

By Zijie Meng, Yufei Liu, Chengqian Ma, Zhiyu Li, Jiyuan Liu, Wenhua Nie, Bingcai Wei, Shuqin Chen, Weichen Xu, Jiquan Yuan, Miao Zhang
arXiv Machine Learning
Jun 9

Echo-Memory: A Controlled Study of Memory in Action World Models

arXiv:2606. 09803v1 Announce Type: cross Abstract: We present \textbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models.

By Wayne King, Zeyue Xue, Yuxuan Bian, Jie Huang, Haoran Li, Yaowei Li, Yaofeng Su, Yuming Li, Haoyu Wang, Shiyi Zhang, Songchun Zhang, Yuwei Niu, Sihan Xu, Junhao Zhuang, Haoyang Huang, Nan Duan