arXiv Computer Vision

IRIS: Implicit Rendering Matters for Pose-Free Novel View Synthesis

arXiv AI
Oct 1

Emergent Multi-View Geometry Through Self-Distillation

The paper introduces Poincar3, a self‑supervised method that learns multi‑view representations through self‑distillation rather than RGB reconstruction. By combining masked patch and image‑level distillation with a teacher that sees additional views, it trains from scratch without explicit 3D supervision. Poincar3 surpasses prior single‑ and multi‑view self‑supervised methods on tasks such as correspondence estimation, camera pose estimation, and 3D reconstruction, and its features encode camera motion more accurately thanks to a lightweight Poincaré adapter.

By David Nordstr\"om, Thibaut Loiseau, Vincent Lepetit, Michael Felsberg, Guillaume Bourmaud, Fredrik Kahl
arXiv Computer Vision
Oct 9

LVSPM: Long Sequence View Synthesis and Pose Estimation Model

LVSPM is a generalizable model that jointly estimates camera poses and synthesizes novel views from uncalibrated image collections. It is trained only with RGB images and pose supervision, avoiding dense 3D ground truth, and uses test‑time training layers to scale to hundreds of input views. On datasets such as RealEstate10k, Co3Dv2, and DL3DV, LVSPM outperforms VGGT in pose estimation across 16–256 views and achieves state‑of‑the‑art pose‑free novel view synthesis quality, maintaining high performance even as scene scale increases.

By Xi Chen, Yachi Zhang, Linghao Chen, Minghua Liu, Hao Su, Zexiang Xu, Xiaoshuai Zhang
arXiv AI
2d ago

Aligning Misaligned Benchmarks for Single-View Novel View Synthesis

The paper addresses the lack of standardized evaluation for single‑view novel view synthesis by highlighting the importance of aligning a method’s reference frame with the ground‑truth camera pose. It proposes using traditional 2D‑to‑2D and 2D‑to‑3D image matching to align 3D reconstructions and restricts evaluation to co‑visible regions where a correct ground truth exists. Experiments show that with proper alignment, 3D Gaussian‑based methods reach state‑of‑the‑art performance, and a simple baseline using a video generator and geometry estimator matches specialized novel‑view synthesis models without task‑specific training.

By Yang-Che Tseng, Erqun Dong, Jeong-gi Kwak, Gopal Sharma, Shakiba Kheradmand, Stephan Garbin, Hossam Isack, Kwang Moo Yi
arXiv Computer Vision
Oct 9

GenIA: Generative Reconstruction with Test-Time Input Alignment

GenIA is a framework that aligns generative 3D foundation models with test‑time observations, improving pose estimation and reconstruction from monocular, multi‑view, and dynamic inputs. It derives translation and scale from geometry, retains learned rotation priors, and aligns appearance using visibility‑biased attention, cross‑observation fusion, and differentiable rendering guidance during denoising. An optional post‑denoising refinement further adapts appearance latents and object placement, and the method also supports externally supplied geometry for dynamic objects, achieving better results than recent optimization‑based and image‑to‑3D methods on synthetic and real benchmarks.

By Stefano Esposito, Naama Pearl, Polina Karpikova, Samuel Rota Bul\`o, Lorenzo Porzi, Peter Kontschieder, Andreas Geiger, Jonathon Luiten
arXiv Computer Vision
Sep 3

RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation

RoGe is a new end‑to‑end framework for novel view synthesis that jointly learns an implicit 3D scene representation and a video diffusion model. It eliminates the need for explicit 3D intermediates by querying the implicit scene with camera rays to produce geometric features that condition the diffusion model. Experiments on DL3DV show that RoGe surpasses reconstruction‑based, generation‑based, and hybrid baselines in image quality and temporal consistency, and ablations confirm the benefits of ray‑queried features and joint training.

By Xiaolei Lang, Ze Kang, Zehao Huang, Naiyan Wang
arXiv Computer Vision
Oct 16

Benchmarking Efficient & Effective Camera Pose Estimation Strategies for Novel View Synthesis

The paper introduces a benchmark for evaluating Structure-from-Motion (SfM) methods used in novel view synthesis (NVS) such as NeRFs and 3DGS. It compares classical SfM, which relies on local feature matches and bundle adjustment, with recent neural network approaches that regress camera parameters directly for faster but less accurate results. The study finds that reducing feature counts speeds up classical SfM without sacrificing pose accuracy, and that combining neural network initial estimates with classical refinement yields the best balance of efficiency and effectiveness.

By Jhacson Meza, Martin R. Oswald, Torsten Sattler