arXiv:2601. 15275v3 Announce Type: replace-cross Abstract: We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene.
We study relative position encoding for multi-view vision Transformers under camera heterogeneity, including varying fields of view (FoVs) or projection models. Existing rotary relative position encod...
arXiv:2610.16337v1 Announce Type: cross Abstract: Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers. In multi-view settings, however, patch coordinates ar...
The paper introduces G-ray, a ray-level relative position encoding for multi-view vision Transformers that remains consistent across different camera projections. By parameterizing rotary phases with camera-local ray angles, G-ray achieves projection-invariant positional consistency and can be integrated with existing encodings without extra learned parameters. Experiments on 3D reconstruction and novel-view synthesis benchmarks show that G-ray improves performance, notably reducing mean pointmap relative error by 45.8% over MapAnything.
The paper introduces a feed‑forward novel view synthesis method that works across heterogeneous cameras, such as perspective, fisheye, and panoramic types. It addresses tokenization ambiguity by combining token‑center relative Camera Positional Encodings with local raymaps and a projection‑aware 2D RoPE that aligns positional reasoning across different camera projections. Experiments on ScanNet++ show improved performance over camera‑conditioned baselines and zero‑shot generalization to panoramic views.
OmniAct3D is a framework that adapts Vision Foundation Model (VFM) detectors trained on perspective images to equirectangular projection (ERP) panoramic images for 3D detection. It introduces the ERP-Ray Geometry Adapter (ERGA-Ray) to model spherical rays, the Visual-Action Reasoning Chain (VARC) to ground hypotheses in panoramic evidence, and the Appearance-Guided Heading Expert (AGHE) to recover high‑resolution heading cues. The method achieves a 2.96 NDS improvement on Spheriverse and a 24.87 mAP gain on PanoMMOcc, while preserving most of the baseline’s performance across configurations.
The paper proposes a new sequence-to-sequence formulation for multi-view stereo (MVS) that jointly predicts 3D geometry for all input views using a global transformer architecture. It introduces ray‑map embeddings to inject camera parameters into image tokens and a unified global cost volume to capture 3D structure across all views. Experiments on public benchmarks demonstrate state‑of‑the‑art performance, outperforming both traditional MVS and feed‑forward reconstruction baselines.
arXiv:2605.12938v2 Announce Type: replace-cross Abstract: Video world models should predict future appearance in a way that remains consistent with 3D scene structure, camera motion, and lens geometr...
arXiv:2512. 16919v2 Announce Type: replace-cross Abstract: Perceiving and reconstructing 3D scene geometry from visual inputs is crucial for autonomous driving.
arXiv:2610.19071v1 Announce Type: new Abstract: Monocular 3D object detection methods increasingly rely on depth-guided transformers, but they operate entirely in the perspective image plane where sp...
arXiv:2606. 31585v1 Announce Type: cross Abstract: The remarkable scalability of Transformers has expanded their application to 3D computer vision, where camera-aware positional encoding is crucial for providing spatial cues in multi-view geometry.
arXiv:2508. 04928v5 Announce Type: replace-cross Abstract: We propose a method to extend foundational monocular depth estimators (FMDEs), trained on perspective images, to fisheye images.