arXiv Computer Vision

Guiding Image-to-3D Generation with Test-Time Partial Observations

arXiv Computer Vision
Oct 15

Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence

arXiv:2610.19502v1 Announce Type: new Abstract: Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominat...

By Yuanyuan Gao, Zhirui Liu, Yuning Gong, Yifei Liu, Yifan Zhan, Kuixuan Jiao, Xiaolong Zhou, Yifan Yang, Hongjie Zhang, Xue Yang, Dan Xu, Zhihang Zhong
arXiv Machine Learning
Sep 4

Sparse auto-regressive modeling for scene generation from multi-view images

The paper introduces SPAR3S, a sparse voxel‑aligned 3D latent generative model that completes 3D scenes from sparse, unconstrained multi‑view images. It learns a compact voxel‑aligned latent space using photometric supervision via differentiable 3D Gaussian Splatting, and employs a masked autoregressive transformer to predict missing voxel occupancy and latent tokens. Experiments on synthetic indoor scenes and RealEstate10k show that SPAR3S achieves higher novel‑view quality than prior methods and generalizes to real‑world data.

By Thomas Lucas, Maxime Pietrantoni, Philippe Weinzaepfel, Wonjune Cho, Bardienus Pieter Duisterhof, Vincent Leroy, Jerome Revaud
arXiv Computer Vision
Sep 3

RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation

RoGe is a new end‑to‑end framework for novel view synthesis that jointly learns an implicit 3D scene representation and a video diffusion model. It eliminates the need for explicit 3D intermediates by querying the implicit scene with camera rays to produce geometric features that condition the diffusion model. Experiments on DL3DV show that RoGe surpasses reconstruction‑based, generation‑based, and hybrid baselines in image quality and temporal consistency, and ablations confirm the benefits of ray‑queried features and joint training.

By Xiaolei Lang, Ze Kang, Zehao Huang, Naiyan Wang
Hugging Face Trending Papers
Sep 3

Sparse auto-regressive modeling for scene generation from multi-view images

The paper introduces SPAR3S, a sparse voxel‑aligned 3D latent generative model that completes 3D scenes from sparse, unconstrained multi‑view images. By representing only occupied voxels in a compact latent space and training a masked autoregressive transformer with photometric supervision via differentiable 3D Gaussian Splatting, the method predicts missing latent tokens and spatial support, enabling efficient and spatially consistent generation of unseen regions. Experiments on synthetic indoor scenes and RealEstate10k demonstrate higher novel‑view quality and real‑world applicability compared to prior work.

arXiv Computer Vision
Aug 24

RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space

RecGen3D is a framework that merges feed‑forward reconstruction and diffusion‑based generation to address the trade‑off between reconstruction fidelity and generative plausibility in sparse‑view 3D modeling. By aligning both models in a shared canonical space and using decoupled cooperative learning, the system stabilizes training and allows the reconstruction module to supply canonical geometric anchors while the diffusion generator refines and completes the structure. Experiments show that RecGen3D outperforms existing methods in producing complete and consistent 3D models from sparse observations.

By Zhisheng Huang, Jiahao Chen, Cheng Lin, Chenyu Hu, Hanzhuo Huang, Zhengming Yu, Mengfei Li, Yuheng Liu, Zekai Gu, Zibo Zhao, Yuan Liu, Xin Li, Wenping Wang
arXiv Computer Vision
Oct 9

GenIA: Generative Reconstruction with Test-Time Input Alignment

GenIA is a framework that aligns generative 3D foundation models with test‑time observations, improving pose estimation and reconstruction from monocular, multi‑view, and dynamic inputs. It derives translation and scale from geometry, retains learned rotation priors, and aligns appearance using visibility‑biased attention, cross‑observation fusion, and differentiable rendering guidance during denoising. An optional post‑denoising refinement further adapts appearance latents and object placement, and the method also supports externally supplied geometry for dynamic objects, achieving better results than recent optimization‑based and image‑to‑3D methods on synthetic and real benchmarks.

By Stefano Esposito, Naama Pearl, Polina Karpikova, Samuel Rota Bul\`o, Lorenzo Porzi, Peter Kontschieder, Andreas Geiger, Jonathon Luiten
arXiv AI
Oct 15

Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective

arXiv:2604.14025v2 Announce Type: replace-cross Abstract: Reconstructing 3D representations from 2D inputs is a fundamental task in computer vision and graphics, serving as a cornerstone for understa...

By Weijie Wang, Qihang Cao, Sensen Gao, Donny Y. Chen, Haofei Xu, Wenjing Bian, Songyou Peng, Tat-Jen Cham, Chuanxia Zheng, Andreas Geiger, Jianfei Cai, Jia-Wang Bian, Bohan Zhuang