Recent video world models have increasingly adopted 3D memory mechanisms to integrate historical observations into a lightweight and persistent scene representation. This naturally raises a complement...
The paper introduces latent spatial memory, a 3D cache that stores scene information directly in diffusion latent space, eliminating the need for pixel-space reconstruction. It presents Mirage, a framework that lifts latent tokens into 3D using depth-guided back‑projection and queries the memory via latent‑space warping, achieving significant speed and memory gains. Experiments demonstrate up to 10.57× faster video generation, a 55× reduction in memory usage, and state‑of‑the‑art performance on WorldScore and strong reconstruction on RealEstate10K.
Video world models that maintain 3D spatial consistency across generated frames typically rely on explicit point cloud memory constructed in RGB space. This design is both computationally expensive, requiring repeated rendering and VAE encoding, and inherently lossy, as the round trip through pixel space discards rich features of the learned latent representation.
The paper introduces the Geometry‑Native Autoencoder (GAE), a compact latent space that can be decoded into appearance, depth, camera parameters, and point maps, enabling 3D‑consistent world generation. By reparameterizing a geometry foundation model’s features, GAE replaces traditional appearance‑centric latents and improves visual quality and 3D coherence, achieving significant reductions in FVD and camera‑trajectory error on benchmark datasets. The work demonstrates that a geometry‑native latent space can serve as a shared interface between perception and generation models.
arXiv:2609.35734v2 Announce Type: replace Abstract: Novel view synthesis from sparse images must reconcile faithful reconstruction of observed regions with plausible completion of unseen content, whi...
RoGe is a new end‑to‑end framework for novel view synthesis that jointly learns an implicit 3D scene representation and a video diffusion model. It eliminates the need for explicit 3D intermediates by querying the implicit scene with camera rays to produce geometric features that condition the diffusion model. Experiments on DL3DV show that RoGe surpasses reconstruction‑based, generation‑based, and hybrid baselines in image quality and temporal consistency, and ablations confirm the benefits of ray‑queried features and joint training.
arXiv:2603. 03482v2 Announce Type: replace-cross Abstract: Interactive world models continually generate video by responding to a user's actions, enabling open-ended generation capabilities.
arXiv:2610.07958v1 Announce Type: new Abstract: Feed-forward 3D Gaussian Splatting (3DGS) reconstructs a scene in a single forward pass, replacing per-scene optimization with a network trained across...
The paper introduces Learning-to-Retrieve (L2R), a method that embeds memory retrieval directly into video world models. By using the model’s internal state as a memory and employing a camera‑conditioned retrieval gate and a retrieval trigger supervised by a 3D re‑visibility signal, the approach learns when and what historical information to access. This eliminates the need for external memory systems and improves long‑term scene consistency across various base models and camera‑revisit benchmarks.
The paper introduces SPAR3S, a sparse voxel‑aligned 3D latent generative model that completes 3D scenes from sparse, unconstrained multi‑view images. By representing only occupied voxels in a compact latent space and training a masked autoregressive transformer with photometric supervision via differentiable 3D Gaussian Splatting, the method predicts missing latent tokens and spatial support, enabling efficient and spatially consistent generation of unseen regions. Experiments on synthetic indoor scenes and RealEstate10k demonstrate higher novel‑view quality and real‑world applicability compared to prior work.
The paper introduces SPAR3S, a sparse voxel‑aligned 3D latent generative model that completes 3D scenes from sparse, unconstrained multi‑view images. It learns a compact voxel‑aligned latent space using photometric supervision via differentiable 3D Gaussian Splatting, and employs a masked autoregressive transformer to predict missing voxel occupancy and latent tokens. Experiments on synthetic indoor scenes and RealEstate10k show that SPAR3S achieves higher novel‑view quality than prior methods and generalizes to real‑world data.
arXiv:2608. 13492v1 Announce Type: new Abstract: This report presents an improved version of AlayaWorld.