arXiv Computer Vision

GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images

GATOR is a generative and agentic framework that reconstructs complete, scene‑aligned 3D objects from casual images. It integrates sparse, uncertain observations by mixing patch‑aligned RGB, target‑mask, and pointmap features, and uses text‑guided semantic conditioning to inform structure, geometry, and appearance generation. The framework then employs a multimodal agent to refine the asset through an observation‑guided edit‑render‑review loop, achieving strong geometric and appearance fidelity across synthetic, cluttered, and indoor scenes while recovering scene‑relative pose from sparse inputs.

arXiv Computer Vision
Aug 26

SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image

SceneReGen is a new framework for reconstructing 3D scenes from a single image by generating and assembling complete object meshes within a shared observation‑aligned scene frame. It uses selective pose factorization to encode each object’s observed orientation directly into the generated mesh, while estimating translation and scale from instance‑level and global scene cues. Evaluated on the 3D‑FUTURE dataset, SceneReGen outperforms existing methods on scene‑level metrics and shows strong performance on object‑level metrics, demonstrating its effectiveness in autonomous‑driving and embodied‑AI scenarios.

By Zefan Tian, Yuteng Ye, Yiheng Zhang, Yuhang Yang, Xueqiang Lv, Shizhou Zhang, Le Liu, Di Xu
arXiv Computer Vision
Sep 23

HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis

HARMONY is a hierarchical chain-of-thought framework that reconstructs complete 3D indoor scenes from a single monocular image. It combines agentic reasoning with visual geometry foundation models, starting with camera calibration and semantic layout recovery, then placing objects hierarchically while refining geometry with point cloud estimations. The method achieves semantically consistent scenes that align perceptually with the input image, outperforming existing baselines on synthetic and real-world data.

By Shufan Sun, Chen Wang, Enxin Song, Jiatao Gu, Lingjie Liu
arXiv Computer Vision
Aug 31

WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild

WilLaGS introduces a unified framework that enhances 3D Gaussian Splatting for in-the-wild scenes by learning a continuous global appearance manifold with a β‑VAE and generating dynamic Tri‑Plane features for spatially‑varying local illumination. It also incorporates a self‑supervised perceptual masking mechanism using a Teacher‑Student EMA architecture to suppress transient artifacts and identify inconsistent regions. Experiments on multiple datasets show that WilLaGS achieves state‑of‑the‑art reconstruction quality and novel view synthesis while preserving real‑time rendering efficiency.

By Yuhao Bai, Qianqiu Tan, Lilong Chen, Huanhuan Lv, Lijun Chen
arXiv Computer Vision
Oct 2

GenCOPE: Syn2Real Generalized Category-Level Object Pose Estimation for Robotic Picking

GenCOPE introduces a synthetic-to-real (Syn2Real) approach for category-level object pose estimation (COPE) that eliminates the need for labor-intensive real-world data collection. By learning domain-invariant representations through 2D and 3D semantic consistency constraints and employing an end-to-end pose regression framework with 2D-3D cross consistency, the model achieves robust generalization across synthetic and real domains. The architecture relies solely on global features, resulting in a lightweight and efficient design validated on REAL275, Wild6D, and real-world robotic manipulation scenes.

By Jian Liu, Wei Sun, Zhenqi Dai, Hui Yang, Jian Xiao, Nicu Sebe, Na Zhao
Hugging Face Trending Papers
Oct 29

LucidLyra: Feed-Forward Generative 3D Scene Reconstruction and Joint Understanding via Multi-Task Distillation

LucidLyra is a framework that reconstructs 3D scenes from video latents in a feed‑forward manner while simultaneously providing open‑vocabulary semantics and instance cues. By distilling knowledge from a camera‑controlled video diffusion model and frozen 2D vision foundation models, it jointly infers 3D Gaussians that encode geometry, appearance, and semantic information without requiring per‑scene optimization or manual 3D annotations. The resulting scenes support novel‑view rendering, open‑vocabulary segmentation, and object grouping, and can be extended to monocular video for dynamic 3D reconstruction.