arXiv Computer Vision

SG-Splat: Stable Geometry Learning under Pose Uncertainty for Pose-Free 3D Gaussian Splatting

arXiv AI
2d ago

Aligning Misaligned Benchmarks for Single-View Novel View Synthesis

The paper addresses the lack of standardized evaluation for single‑view novel view synthesis by highlighting the importance of aligning a method’s reference frame with the ground‑truth camera pose. It proposes using traditional 2D‑to‑2D and 2D‑to‑3D image matching to align 3D reconstructions and restricts evaluation to co‑visible regions where a correct ground truth exists. Experiments show that with proper alignment, 3D Gaussian‑based methods reach state‑of‑the‑art performance, and a simple baseline using a video generator and geometry estimator matches specialized novel‑view synthesis models without task‑specific training.

By Yang-Che Tseng, Erqun Dong, Jeong-gi Kwak, Gopal Sharma, Shakiba Kheradmand, Stephan Garbin, Hossam Isack, Kwang Moo Yi
arXiv Computer Vision
Oct 21

CrossFit-GS: Out-of-Fold Pose Estimation for COLMAP-Free 3D Gaussian Splatting

CrossFit-GS is a new framework for pose-free 3D Gaussian Splatting that avoids the need for Structure-from-Motion by directly reconstructing scenes from unposed image sequences. The method partitions the input into interleaved subsets, maintaining two complementary submaps and cross-refining candidate poses against a frozen opposite submap to provide an independent geometric reference. Additional checkerboard validation and spatial coverage safeguards reject unreliable updates, and only verified poses are committed; after refinement, the final global representation is trained from scratch using the fixed refined poses, leading to reduced trajectory drift and improved camera pose estimation and novel-view synthesis across multiple benchmarks.

By Zijian Wu, Jinliang Wang, Mingfeng Jiang, Zidian Lin, Ying Song, Ziqian Lu
arXiv Computer Vision
Sep 21

VoxelTTO: Voxel-Aligned Feed-Forward 3D Gaussian Splatting with Test-Time Optimization

VoxelTTO is a feed‑forward framework that reconstructs 3D Gaussian splatting scenes from multiple images by aggregating dense image features into a global voxel representation and decoding Gaussians from voxel features, thereby eliminating the pixel‑to‑Gaussian correspondence. It incorporates test‑time optimization with lightweight LoRA modules to adapt to known camera parameters while keeping the pretrained visual foundation model frozen. The method replaces standard rasterization with stochastic solid volume rendering, improving geometric fidelity, and demonstrates superior RGB‑D novel‑view synthesis and camera‑pose estimation on Replica, Tanks and Temples, and DTU datasets.

By Yibin Zhao, Yihan Pan, Yangwen Li, Jun Nan, Jianjun Yi
arXiv Computer Vision
Oct 12

NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement

NovaSplat is a model‑free pipeline for 6D object pose estimation that builds a 3D Gaussian Splatting (3DGS) model from a short smartphone video during onboarding. The 3DGS model serves both as a template for detection and coarse pose estimation and as a differentiable backbone for pose refinement, which optimizes pose using photometric and geometric losses under a continuous rotation parametrization. The method outperforms all model‑based methods on a household‑object benchmark and all model‑free competitors on a challenging industrial benchmark, while a faster variant, NovaSplat‑Lite, achieves comparable accuracy with a 3.1× speedup.

By Georgios Floros, Evangelos Sartinas, Orestis Vangelis, Maria Pateraki
arXiv Computer Vision
Aug 27

PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

PIVOT is a new multi‑trajectory dataset and evaluation framework that captures real‑world scenes with diverse camera paths, preserving both sensor‑derived measured poses and COLMAP‑optimized poses along with calibrated and optimized intrinsics. It defines three benchmark families—seen vs. unseen trajectory generalization, measured vs. optimized pose sensitivity, and calibrated vs. optimized intrinsics sensitivity—and introduces a directed pose‑space Chamfer distance to assess pose coverage. The first version of PIVOT includes five scenes recorded with a DJI Mini 4 Pro and offers an open processing and Nerfstudio‑based evaluation toolchain, revealing a consistent quality gap between held‑out and unseen trajectories and significant sensitivity to pose source and camera intrinsics.

By Mary Raymond