Multi-View Foundation Models
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The paper introduces Z3D, a method that leverages internal representations from 3D Foundation Models (3DFMs) to perform zero‑shot novel depth synthesis. By decoding hidden surfaces and applying latent diffusion on 3DFM representations, Z3D can estimate realistic depth maps for unseen views across multiple datasets. This demonstrates that 3DFMs capture extensive general knowledge about 3D scenes, enabling accurate reconstruction without additional training.
arXiv:2608.23850v1 Announce Type: new Abstract: Foundational visual features such as DINO have played a critical role across modern computer vision, and have recently become key components in multi-v...
arXiv:2609.01172v1 Announce Type: new Abstract: Monocular depth estimation has long stood as a fundamental challenge in computer vision, enabling a wide range of applications including 3D reconstruct...
The paper introduces a generalizable deformation learning framework that reconstructs 3D objects by deforming a category-level shape template to match a monocular observation. It employs a geometry-guided feature modeling mechanism to enrich foundation features with template topology, creating a geometry-aware representation that is explicitly correlated with the target observation for precise deformation. A view-adaptive feature aggregation module further bridges the gap between the fixed template and arbitrary target views by leveraging multi-view template features and camera poses, ensuring robust feature alignment across diverse viewpoints.
The paper explores how 3D Foundation Models (3DFMs) like VGGT can be leveraged for zero‑shot depth synthesis. By decoding hidden surfaces from the models’ internal representations, the authors introduce Z3D, a method that uses latent diffusion on 3DFM representations to estimate pointmaps in unseen views. Experiments demonstrate that Z3D can generate realistic depth maps across multiple datasets.
The paper introduces Epipolar Distillation (EpiDistill), a method that transfers scale‑aware geometric priors from multi‑view models to monocular depth foundation models using Rectified Stereo Tokens. By preserving epipolar attention patterns, the single‑view model maintains geometric consistency without needing multi‑view inputs during inference. Experiments show significant improvements in zero‑shot metric depth estimation on challenging datasets such as ETH3D and DIODE, and the approach consistently boosts performance of state‑of‑the‑art ViT‑based models like UniDepthV2 and DepthPro.