arXiv Computer Vision

OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping

OctWorld is a video diffusion framework that uses a persistent 3D memory called OctMap to generate long‑range, world‑consistent videos from a single image. OctMap fuses visual observations and depth maps into a dynamic sparse octree, adapting spatial resolution to image evidence while keeping memory usage low. Experiments show that OctWorld produces spatially consistent videos over extended camera paths and outperforms previous methods on both standard benchmarks and challenging long‑range scenarios.

Hugging Face Trending Papers
Sep 3

OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping

OctWorld is a video diffusion framework that uses a persistent 3D memory called OctMap to generate long‑range, world‑consistent videos from a single image. OctMap fuses visual observations and depth maps into a dynamic sparse octree with adaptive resolution, preserving geometric and appearance details while keeping memory usage low. Experiments show OctWorld outperforms previous methods on standard benchmarks and challenging long‑range generation tasks.

arXiv Computer Vision
Oct 9

WorldWeave: Growing Persistent Geometric Worlds for Video Generation

WorldWeave is a framework that separates world-state maintenance from visual rendering to enable persistent geometric memory in video generation. It uses diffusion-based image outpainting to create continuous elevation maps, an agent that plans and stitches scenes hierarchically, and a read-only interface that queries world geometry for depth-guided video synthesis. This design allows the world to expand continually without predefined boundaries while preserving consistent structure across camera trajectories and revisits.

By Yifan Huang, Lifan Jiang, Qingyue Hao, Cheng Chen, Boxi Wu, Xiaoxue Ren, Xiaofei He, Dehai Zhao
arXiv Computer Vision
Nov 5

EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory

EvoWorld is a world model that generates panoramic video from a single image by combining a video generator with fine‑grained view control, a feedforward transformer that evolves an explicit 3D reconstruction, and a synthesis step that conditions on geometric reprojections from this evolving 3D memory. The model uses the evolving 3D reconstruction as spatial guidance, projecting geometry onto target viewpoints to provide rich spatial cues that improve visual realism and geometric consistency. A new benchmark covering synthetic outdoor, Habitat indoor, and real‑world scenarios evaluates long‑range exploration, loop‑closure detection, and spatial coherence, showing that EvoWorld’s evolving 3D memory enhances visual fidelity and scene coherence over existing methods.

By Jiahao Wang, Luoxin Ye, TaiMing Lu, Junfei Xiao, Jiahan Zhang, Yuxiang Guo, Xijun Liu, Rama Chellappa, Cheng Peng, Alan Yuille, Jieneng Chen
arXiv Computer Vision
Oct 19

Persistent Planar Memory for Video World Models

Persistent Planar Memory for Video World Models (PPMem) introduces a compact 3D memory system using plane primitives for video world models. By integrating a planar prediction head and Depth Token Fusion into a video diffusion transformer, PPMem jointly generates RGB video and planar geometry, merging them into a persistent plane memory that conditions future video chunks. The resulting memory is highly compact, geometrically explicit, cross-view consistent, and adaptive, outperforming prior 3D-grounded models on DL3DV and RealEstate10K in video quality and depth/normal accuracy.

By Yuze He, Bin Tan, Zelin Gao, Yujun Shen, Yong-Jin Liu, Nan Xue, Yinghao Xu
arXiv Computer Vision
Aug 28

SpatialCrafter: Single Image World Modeling with Generative 3D Proxies

SpatialCrafter introduces a two‑stage framework for single‑image world modeling that first generates a global 3D proxy using a Point‑anchored Sparse Structure Flow module, then refines appearance with a Generative Deferred Refiner built on a video diffusion model. The method incorporates Parallel Geometry Injection and Proxy‑Aware Corruption training to integrate the proxy without disrupting the pretrained generative manifold, and it is evaluated on a newly constructed dataset of 115K scenes. Experiments demonstrate that SpatialCrafter outperforms existing approaches, reducing long‑term drift and maintaining consistency under rapid camera motion and extreme viewpoints.

By Chuan Fang, Lingteng Qiu, Yixun Liang, Rui Chen, Kunming Luo, Zhaohua Zheng, Tongyuan Bai, Feipeng Tian, Zilong Dong, Zihan Zhou, Ping Tan
arXiv Computer Vision
Aug 28

Latent Spatial Memory for Video World Models

The paper introduces latent spatial memory, a 3D cache that stores scene information directly in diffusion latent space, eliminating the need for pixel-space reconstruction. It presents Mirage, a framework that lifts latent tokens into 3D using depth-guided back‑projection and queries the memory via latent‑space warping, achieving significant speed and memory gains. Experiments demonstrate up to 10.57× faster video generation, a 55× reduction in memory usage, and state‑of‑the‑art performance on WorldScore and strong reconstruction on RealEstate10K.

By Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y. Chen, Yuqing Yang, Bohan Zhuang
arXiv Computer Vision
Nov 2

VoxLution: Evolving Voxels for High-Fidelity 3D Scene Generation

VoxLution is a coarse‑to‑fine transformer‑based method for generating high‑fidelity 3D indoor scenes. It first creates a low‑resolution spatial layout and then progressively refines the geometry in local chunks using a sequence‑modeling approach over multi‑resolution sparse voxel grids. The method introduces a resolution‑invariant tokenization scheme that encodes octants into a single decimal number and incorporates adjacent chunk boundaries to maintain coherence, achieving state‑of‑the‑art performance on 3D‑FRONT and ScanNet++ benchmarks.

By Daoyi Gao, Xinyi Zhang, Alexey Bokhovkin, Lei Li, Angela Dai