arXiv Computer Vision

VI3: Grounding Pretrained 3D Foundation Models with Inertial Cues

VI3 is a model‑agnostic framework that grounds pretrained 3D foundation models (3DFMs) by using inertial measurement unit (IMU) data to provide metric scale. It initializes and preintegrates IMU readings to create a metric motion reference, which is then used to recover the scale of 3DFM outputs. The approach includes adaptable anchoring strategies for different 3DFM architectures and demonstrates scale recovery on synthetic and real aerial datasets without ground‑truth supervision.

arXiv AI
Jul 7

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

arXiv:2601. 22054v2 Announce Type: replace-cross Abstract: Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data.

By Baorui Ma, Jiahui Yang, Donglin Di, Xuancheng Zhang, Jianxun Cui, Hao Li, Yan Xie, Wei Chen
Hugging Face Trending Papers
Jun 29

Argus: Metric Panoramic 3D Reconstruction for Indoor Scenes

Metric feed-forward 3D reconstruction for panoramic data remains under-explored due to the lack of large-scale panoramic RGB-D training data. We present Realsee3D, a hybrid dataset of 10K indoor scenes (1K real, 9K synthetic) with 299K panoramic viewpoints and precise metric annotations, and Argus, a feed-forward network trained on it for metric panoramic 3D reconstruction.

Hugging Face Trending Papers
Jun 1

Honey, I Shrunk the Arc de Triomphe!

Metric scale monocular geometry estimation has seen significant progress through large-scale data aggregation, yet current foundation models suffer from a persistent ''scale-collapse'' phenomenon: distant landmarks and vast landscapes are metrically underestimated. We hypothesize that this performance gap stems from a training data bottleneck, where existing metric-scale datasets are hardware-constrained to homogenous vehicle-captured LiDAR or short-range indoor scans, or consist of synthetic data that lacks the semantic complexity of the physical world.

arXiv Computer Vision
Sep 3

Geometric Distillation from Rectified Stereo: Leveraging Epipolar Cues for Monocular Depth

The paper introduces Epipolar Distillation (EpiDistill), a method that transfers scale‑aware geometric priors from multi‑view models to monocular depth foundation models using Rectified Stereo Tokens. By preserving epipolar attention patterns, the single‑view model maintains geometric consistency without needing multi‑view inputs during inference. Experiments show significant improvements in zero‑shot metric depth estimation on challenging datasets such as ETH3D and DIODE, and the approach consistently boosts performance of state‑of‑the‑art ViT‑based models like UniDepthV2 and DepthPro.

By Jung-Hee Kim, Xiaoming Liu
arXiv Computer Vision
Sep 4

Zero-Shot Novel Depth Synthesis Using 3D Foundation Models Scene Representations

The paper introduces Z3D, a method that leverages internal representations from 3D Foundation Models (3DFMs) to perform zero‑shot novel depth synthesis. By decoding hidden surfaces and applying latent diffusion on 3DFM representations, Z3D can estimate realistic depth maps for unseen views across multiple datasets. This demonstrates that 3DFMs capture extensive general knowledge about 3D scenes, enabling accurate reconstruction without additional training.

By Denis M. Akola, David F. Fouhey
arXiv Computer Vision
Sep 3

Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models

The paper introduces Self-Geometry, a plug‑and‑play test‑time adaptation framework that enforces explicit multi‑view geometric constraints on Vision Foundation Models (VFMs) using 2D pixel correspondences as pseudo ground truth. It combines Geometric Disentanglement Optimization—mixing Multi‑View and Epipolar Consistency losses with Gradient Disentanglement—to avoid gradient conflicts, a Frame Angular‑Neighbor sampler based on SO(3) geodesic distances to select informative views, and a Lightweight TTA module that adapts VFMs via LoRA. Experiments on six VFMs and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom) show consistent improvements in pose and geometry estimation.

By Seokhyun Youn, Dahyeon Kye, Sung-Ho Bae, Jihyong Oh