arXiv Computer Vision

SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images

arXiv Computer Vision
Nov 6

Bri3D: Bridging the Object-Scene Gap in Single-Image 3D Generation via Scene-Anchor

Bri3D is a new framework for generating compositional 3D scenes from a single image. It bridges scene-level structural reasoning with object-level generative priors by using instance-aware voxel scene anchors that encode coarse geometry and spatial arrangements. The method refines each anchor into a pose-preserving voxel support, allowing a frozen object generator to produce detailed, textured geometry while separating scene organization from object-scale detail synthesis, resulting in faster and higher-quality scene generation compared to prior works.

By Hongye Hou, Shuai Zhang, Hongjie Li, Zhuoxiao Li, Haiping Wang, Yuan Liu, Wufan Zhao
arXiv Computer Vision
Aug 26

SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image

SceneReGen is a new framework for reconstructing 3D scenes from a single image by generating and assembling complete object meshes within a shared observation‑aligned scene frame. It uses selective pose factorization to encode each object’s observed orientation directly into the generated mesh, while estimating translation and scale from instance‑level and global scene cues. Evaluated on the 3D‑FUTURE dataset, SceneReGen outperforms existing methods on scene‑level metrics and shows strong performance on object‑level metrics, demonstrating its effectiveness in autonomous‑driving and embodied‑AI scenarios.

By Zefan Tian, Yuteng Ye, Yiheng Zhang, Yuhang Yang, Xueqiang Lv, Shizhou Zhang, Le Liu, Di Xu
arXiv AI
Aug 25

Object-Uni: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation

Object-Uni is a unified model that integrates pose perception, spatial reasoning, pose-conditioned generation, and novel view synthesis for object-centric spatial understanding and controllable image generation. It treats object pose as an explicit geometric variable shared across tasks and introduces a viewpoint-based orientation abstraction to make pose interpretable by multimodal large language models. The authors also create a new benchmark, UniSpatial-80K, and demonstrate that Object‑Uni improves both pose understanding and pose‑controllable generation compared to existing models.

By Mining Tan, Yinuo Wang, Ziqi Zhou, Weize Quan, Sifei Li, Jingdong Chen, DanDan Zheng, Libin Wang, Weiming Dong
arXiv AI
Oct 26

Easy-3Dfy: Object-Level Geometric Composition for Single-Image 3D Scene Construction

Easy-3Dfy introduces a compositional workflow for single-image 3D scene construction that separates object generation from scene layout estimation. By using a specialized 3D object generator for detailed geometry and a 3D foundation model for spatial understanding, the method aligns object-local meshes with partial scene pointmaps via an RPF-based aligner. This approach consistently outperforms SAM-3Dfy on geometric metrics in Scene-Maker and 3D-FUTURE200 without requiring full scene-level supervision.

By Ziang Zhang, Wenyou Yi, Zehan Wang, Jiachen Xu, Xintong Han, Chunchao Guo, Zhou Zhao
arXiv Machine Learning
6d ago

UGG: Unified Generative Geometry Model

UGG is a unified generative geometry model that jointly learns RGB video and geometric representations such as depth and camera pose. By assigning different noise levels across modalities and time, it treats visual generation and geometric prediction as conditional completions within a single generative framework. This approach allows learning from heterogeneous datasets, supports inference-time search over multiple scene hypotheses, and outperforms non-unified baselines in tasks like camera-controlled video generation, depth estimation, and camera pose prediction, even with sparse observations.

By Amani Kiruga, Zhiyi Li, Ruojin Cai, Hansen Lillemark, Xiaoshan Wu, Ayush Tewari, Yilun Du
arXiv Computer Vision
Sep 7

WorldSculpt: Generating Compositional Worlds from Grounded Videos

WorldSculpt presents a method for generating compositional 3D representations of cluttered scenes with hundreds of objects by adapting a single-object 3D generative prior to multi-view observations. The approach, built on Pixal3D with a multi-view conditioning pathway, can generalize to highly occluded scenes without scene-level training. The authors also introduce the UE-MeshyScene benchmark and demonstrate that their method outperforms prior approaches across various evaluation settings, including converting existing 3DGS worlds into compositional mesh scenes.

By Muyao Niu, Jixuan He, Ruihan Yu, Lian Fu, Yonghao Yu, Zheng-Hui Huang, Yifan Zhan, Fengbo Lan, Yongtao Ge, Yinqiang Zheng, Kaipeng Zhang, Zhixiang Wang
arXiv AI
Oct 31

Aligning Misaligned Benchmarks for Single-View Novel View Synthesis

The paper addresses the lack of standardized evaluation for single‑view novel view synthesis by highlighting the importance of aligning a method’s reference frame with the ground‑truth camera pose. It proposes using traditional 2D‑to‑2D and 2D‑to‑3D image matching to align 3D reconstructions and restricts evaluation to co‑visible regions where a correct ground truth exists. Experiments show that with proper alignment, 3D Gaussian‑based methods reach state‑of‑the‑art performance, and a simple baseline using a video generator and geometry estimator matches specialized novel‑view synthesis models without task‑specific training.

By Yang-Che Tseng, Erqun Dong, Jeong-gi Kwak, Gopal Sharma, Shakiba Kheradmand, Stephan Garbin, Hossam Isack, Kwang Moo Yi