arXiv Computer Vision

GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation

The paper introduces the Geometry‑Native Autoencoder (GAE), a compact latent space that can be decoded into appearance, depth, camera parameters, and point maps, enabling 3D‑consistent world generation. By reparameterizing a geometry foundation model’s features, GAE replaces traditional appearance‑centric latents and improves visual quality and 3D coherence, achieving significant reductions in FVD and camera‑trajectory error on benchmark datasets. The work demonstrates that a geometry‑native latent space can serve as a shared interface between perception and generation models.

arXiv Computer Vision
Oct 21

Latent Bridge: From Video Latents to 3D Geometry via Feature Alignment

Latent Bridge connects video generative latent spaces with geometric feature spaces, enabling direct prediction of 3D geometry—cameras, depth, and point clouds—without an intermediate RGB decoding step. The method trains representation‑specific bridges for Wan2.1 and V‑RAE via feature distillation on RGB videos while keeping the video encoder and geometry model frozen, requiring no ground‑truth geometry. At inference, the same video latents support both RGB decoding and geometry prediction, and the trained bridge can be reused across image‑to‑video and text‑to‑video generation, achieving competitive reconstruction quality on Co3Dv2 and WildRGB‑D with efficient end‑to‑end generation.

By Shuai Zhang, Zhanpeng Ouyang, Zhengqing Chen, Ruohong Mei, Shaoyu Chen, Weiqiang Ren, Hui Sun, Qian Zhang, Wenyu Liu, Xinggang Wang
arXiv Computer Vision
Nov 9

Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models

The paper introduces Latent Riemannian Flow Matching, a method that performs flow matching directly in the latent space of geometric foundation models like VGGT. By respecting the product of zero‑mean hyperspheres imposed by LayerNorm, the approach avoids the pitfalls of treating the latent space as unconstrained Euclidean. Experiments on RealEstate10K, ScanNet++, and ETH3D show that this technique outperforms recent scene‑generation baselines in both novel‑view appearance and aggregated 3D geometry, and it transfers to other pretrained 3D backbones such as Depth Anything 3.

By Lisa Weijler, Irene Ballester, Guofeng Mei, Tolga Birdal, Pedro Hermosilla
arXiv Computer Vision
Oct 28

Learning and Reasoning with Geometry: How Generation Helps Spatial Understanding

The paper introduces GeoUG, a unified model that combines depth and multi-view generation with spatial understanding. It employs a multi-scale depth codec for metric-depth generation and demonstrates that joint training improves spatial tasks. Using the 3DCueBench benchmark, the authors show that generated depth aids fine-grained geometric reasoning and develop an adaptive interleaved reasoning strategy that balances accuracy with generation cost, achieving high accuracies on several benchmarks.

By Rui Yang, Yanwei Li, Sili Chen, Jingjia Huang, Yi Lin, Hengshuang Zhao
arXiv Machine Learning
6d ago

UGG: Unified Generative Geometry Model

UGG is a unified generative geometry model that jointly learns RGB video and geometric representations such as depth and camera pose. By assigning different noise levels across modalities and time, it treats visual generation and geometric prediction as conditional completions within a single generative framework. This approach allows learning from heterogeneous datasets, supports inference-time search over multiple scene hypotheses, and outperforms non-unified baselines in tasks like camera-controlled video generation, depth estimation, and camera pose prediction, even with sparse observations.

By Amani Kiruga, Zhiyi Li, Ruojin Cai, Hansen Lillemark, Xiaoshan Wu, Ayush Tewari, Yilun Du