arXiv Computer Vision

Depth Anything in $360^\circ$: Towards Scale Invariance in the Wild

arXiv AI
6d ago

Unified Panoramic Geometry Estimation via Multi-View Foundation Models

The paper introduces PaGeR, a framework that adapts pre‑trained 3D reconstruction transformers to handle both perspective and panoramic images. PaGeR predicts scale‑invariant depth, metric depth, surface normals, and sky masks in a single forward pass, using minimal architectural changes and mixed‑domain training. Extensive indoor and outdoor tests show state‑of‑the‑art and strong zero‑shot performance across diverse scenes.

By Vukasin Bozic, Isidora Slavkovic, Dominik Narnhofer, Nando Metzger, Denis Rozumny, Konrad Schindler, Nikolai Kalischek
arXiv Computer Vision
Oct 15

Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence

arXiv:2610.19502v1 Announce Type: new Abstract: Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominat...

By Yuanyuan Gao, Zhirui Liu, Yuning Gong, Yifei Liu, Yifan Zhan, Kuixuan Jiao, Xiaolong Zhou, Yifan Yang, Hongjie Zhang, Xue Yang, Dan Xu, Zhihang Zhong
arXiv Computer Vision
Nov 6

PVDepth: Panoramic Video Depth Estimation via Geometry-Aware Spatiotemporal Adaptation

The paper introduces PVDepth, an end‑to‑end framework for panoramic video depth estimation that adapts perspective video depth models to the equirectangular projection (ERP) domain. It first presents PanoCARLA, a large‑scale synthetic RGB‑D panoramic video dataset with natural motion trajectories and drone‑like viewpoints. PVDepth addresses ERP‑specific challenges with a Progressive Sphere‑aware Noise Initialization strategy and a Cube‑rectified Temporal Modeling module, achieving superior geometrically accurate and temporally consistent depth sequences.

By Chuanxin Song, Peixi Peng
arXiv AI
Nov 6

PanoWorld: Geometry-Consistent Panoramic Video World Modeling

PanoWorld is a panoramic video world model that generates geometry‑consistent 360° video from a single image and a caption. It frames panoramic video generation as a geometry‑ and dynamics‑consistent latent state modeling problem, using depth consistency and trajectory consistency regularizers built on a pre‑trained perspective video world model. The authors also introduce PanoGeo, a unified geometry‑aware panoramic video dataset with consistent depth, trajectory, and prompt annotations, and demonstrate that PanoWorld improves geometric consistency while maintaining competitive visual realism.

By Le Jiang, Xiangyu Bai, Bishoy Galoaa, Shayda Moezzi, Caleb James Lee, Tooba Imtiaz, Edmund Yeh, Jennifer Dy, Yanzhi Wang, Sarah Ostadabbas
arXiv Computer Vision
Oct 2

Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models

Dyna3 is a training‑free framework that extends the depth foundation model DA3 to perform 4D dynamic scene reconstruction without fine‑tuning. By leveraging DA3’s cross‑view features and a best‑match search, it distinguishes static surfaces from moving objects, and uses vision‑language models to generate semantic prompts for SAM 3 to achieve precise instance‑level segmentation. Experiments on four datasets show Dyna3 outperforms correspondence‑trained methods, improving dynamic object segmentation by +5.5 pp, speeding pose estimation 13×, and reducing memory usage 4–8×.

By Xinhao Xiang, Weiyang Li, Zhijie Zheng, Abhijeet Rastogi, Jiawei Zhang
arXiv Computer Vision
Sep 3

Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models

The paper introduces Self-Geometry, a plug‑and‑play test‑time adaptation framework that enforces explicit multi‑view geometric constraints on Vision Foundation Models (VFMs) using 2D pixel correspondences as pseudo ground truth. It combines Geometric Disentanglement Optimization—mixing Multi‑View and Epipolar Consistency losses with Gradient Disentanglement—to avoid gradient conflicts, a Frame Angular‑Neighbor sampler based on SO(3) geodesic distances to select informative views, and a Lightweight TTA module that adapts VFMs via LoRA. Experiments on six VFMs and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom) show consistent improvements in pose and geometry estimation.

By Seokhyun Youn, Dahyeon Kye, Sung-Ho Bae, Jihyong Oh
arXiv AI
3d ago

Multi-View 3D Reconstruction Using V-JEPA Features

The paper investigates supervised RGB‑only multi‑view 3D reconstruction by integrating video‑pretrained features with camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, reporting improvements in trajectory and intrinsics errors but mixed effects on depth accuracy and raw Chamfer distance. The results emphasize that representation adaptation, shape alignment, and raw reconstruction must be evaluated separately before attributing gains to predictive video pretraining.

By Anwar Shamim, Jeffrin Sam
arXiv AI
Jul 7

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

arXiv:2601. 22054v2 Announce Type: replace-cross Abstract: Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data.

By Baorui Ma, Jiahui Yang, Donglin Di, Xuancheng Zhang, Jianxun Cui, Hao Li, Yan Xie, Wei Chen