arXiv Computer Vision

Measurable Semantic 3D Gaussian Splatting with Joint Region and Depth Supervision for Interactive Building Inspection

The paper introduces a measurable semantic 3D Gaussian Splatting (3DGS) framework that uses region and depth supervision to improve building inspection from multi‑view imagery. Mask‑based region supervision focuses optimization on target areas, while point‑wise depth supervision aligns Gaussian centers with physical surfaces derived from multi‑view stereo point clouds. The resulting semantic Gaussian field enables component‑level inspection and is deployed in the AEC‑SplatTwin platform, achieving significant reductions in depth error and high semantic segmentation accuracy on UAV‑inspected and smartphone‑captured datasets.

arXiv Computer Vision
Sep 3

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

PointGauss is a 3D-native framework that performs semantic parsing and instance segmentation on 3D Gaussian splatting representations by treating Gaussian primitives as unstructured point sets and extracting scale‑invariant geometric features with Point Transformer V3. It introduces an adaptive region‑of‑interest cropping strategy and an instance‑aware distance‑constrained rasterization pipeline to enable scalable, view‑consistent pixel‑level projections. The authors also release SplatSeg‑360, a cross‑scale benchmark with 32 complex scenes and over 6,300 aligned 2D‑3D masks, and show that PointGauss achieves real‑time performance with state‑of‑the‑art 3D‑mIoU (~90%) and 2D‑mIoU (~80%) scores.

By Wentao Sun, Yiping Chen, John S. Zelek, Jonathan Li
arXiv Computer Vision
Sep 24

GaussianDS: Depth-supervised Semantic Gaussian Splatting for Scene Understanding

GaussianDS introduces a depth‑supervised framework for 3D Gaussian Splatting that jointly optimizes RGB appearance, depth, and compact semantics from scratch. By arranging multi‑view images into a pose‑aware pseudo‑video and propagating view‑consistent masks via SAM2, the method aligns semantic lifting with geometric cues, using depth supervision and edge‑aware refinement to curb semantic drift and boundary leakage. The approach achieves state‑of‑the‑art performance on LERF and 3D‑OVS benchmarks while preserving high‑fidelity reconstruction and enabling downstream tasks such as 3D object removal.

By Yufei Zhang, Chenlu Zhan, Hongwei Wang
arXiv Computer Vision
Oct 2

MEGA: Object-Level Mesh Extraction from 3D Gaussian Splatting via Spatial Visual Distillation

MEGA is a new framework that extracts object-level, watertight meshes from 3D Gaussian Splatting (3DGS) scenes. It uses a segment-then-mesh approach, leveraging Spatial Visual Distillation (SVD) to sample diverse camera views of each segmented object and train a mesh reconstruction model with photometric supervision. Experiments on popular benchmarks show that MEGA outperforms existing methods in accurately recovering object-level 3D occupancy and supports complex physical interactions by combining high-quality meshes with photorealistic 3DGS rendering.

By Liwei Liao, Yingkui Zhang, Qianqian Tong, Ronggang Wang
arXiv AI
Oct 28

GCSR-OCC: Geometry Calibration and Semantic Routing for Open-Vocabulary Occupancy Prediction

GCSR‑OCC introduces geometry calibration and semantic routing to improve open‑vocabulary 3D occupancy prediction from a single RGB image. It uses an Evidence‑Conserving Gaussian‑to‑Occupancy operator to mitigate scale inflation and a Multi‑Mode Query‑First Gaussian Semantic Routing module to align multiple semantic modes with text queries, refining voxel‑level predictions. The method achieves state‑of‑the‑art results on Occ‑ScanNet, improving geometric IoU to 63.31% and semantic mIoU to 27.44%, while reducing aggregation latency from 19.76 ms to 12.88 ms.

By Bohua Wang, Congyang Zhao, Tianxiong Lv, Chenxu Zhao, He Zheng, Hangqi Fan, Leigang Luo, Zhihao Liao, Yihong Guo, Yang Cai
arXiv Computer Vision
Oct 12

Domain Adaptation of SAM 2.1 for Building-Component Segmentation in Image-Based 3D Reconstruction

The paper reports fine‑tuning the Segment Anything Model (SAM) 2.1 on the HBD dataset of 2,679 images from 42 buildings to improve multi‑class segmentation of 2D building images. The fine‑tuned model achieves a higher mean Intersection‑over‑Union (mIoU) of 69.6% compared to 49.9% for the pretrained version, and shows better performance on larger object classes. These enhanced 2D masks are then used in Group Anything with Radiance Fields (GARField) to group building components more accurately in 3D neural radiance fields, facilitating more precise 3D auto‑reconstruction of building structures.

By Ehsan Agha Ebrahimi, Yong-Cheol Lee, Keith G. Mills
arXiv Computer Vision
Sep 16

DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding

DGSG-Mind introduces a hybrid instance-aware 3D Gaussian dynamic scene graph system that integrates open‑vocabulary semantic information into dynamic 3D scene representations. By coupling a probabilistic voxel grid with explicit 3D Gaussians, it achieves robust cross‑modal instance fusion, incremental semantic mapping, and dynamic change handling through Gaussian‑based relocalization and masked refinement. The system builds a hierarchical scene graph and a 3D Gaussian Mind for multimodal reasoning, achieving state‑of‑the‑art zero‑shot 3D visual grounding and strong performance in open‑vocabulary semantic segmentation and scene reconstruction, and is demonstrated on real‑world robots.

By Luzhou Ge, Xiangyu Zhu, Jinyan Liu, Xuesong Li