arXiv Computer Vision By Yuze He, Bin Tan, Zelin Gao, Yujun Shen, Yong-Jin Liu, Nan Xue, Yinghao Xu

Persistent Planar Memory for Video World Models

Read the original on arXiv Computer Vision →

Persistent Planar Memory for Video World Models (PPMem) introduces a compact 3D memory system using plane primitives for video world models. By integrating a planar prediction head and Depth Token Fusion into a video diffusion transformer, PPMem jointly generates RGB video and planar geometry, merging them into a persistent plane memory that conditions future video chunks. The resulting memory is highly compact, geometrically explicit, cross-view consistent, and adaptive, outperforming prior 3D-grounded models on DL3DV and RealEstate10K in video quality and depth/normal accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Sep 4

OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping

OctWorld is a video diffusion framework that uses a persistent 3D memory called OctMap to generate long‑range, world‑consistent videos from a single image. OctMap fuses visual observations and depth maps into a dynamic sparse octree, adapting spatial resolution to image evidence while keeping memory usage low. Experiments show that OctWorld produces spatially consistent videos over extended camera paths and outperforms previous methods on both standard benchmarks and challenging long‑range scenarios.

By Zelong Lv, Sicheng Xu, Jianfeng Xiang, Ruicheng Wang, Yue Dong, Yu Deng, Guangzhong Sun, Jiaolong Yang
arXiv Computer Vision
Aug 28

Latent Spatial Memory for Video World Models

The paper introduces latent spatial memory, a 3D cache that stores scene information directly in diffusion latent space, eliminating the need for pixel-space reconstruction. It presents Mirage, a framework that lifts latent tokens into 3D using depth-guided back‑projection and queries the memory via latent‑space warping, achieving significant speed and memory gains. Experiments demonstrate up to 10.57× faster video generation, a 55× reduction in memory usage, and state‑of‑the‑art performance on WorldScore and strong reconstruction on RealEstate10K.

By Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y. Chen, Yuqing Yang, Bohan Zhuang
Hugging Face Trending Papers
Sep 3

OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping

OctWorld is a video diffusion framework that uses a persistent 3D memory called OctMap to generate long‑range, world‑consistent videos from a single image. OctMap fuses visual observations and depth maps into a dynamic sparse octree with adaptive resolution, preserving geometric and appearance details while keeping memory usage low. Experiments show OctWorld outperforms previous methods on standard benchmarks and challenging long‑range generation tasks.