arXiv Computer Vision

Rohbau3D-Semantics: Dense Semantic and Instance Labels for Point Clouds of Indoor Shell Construction Scenes

arXiv Computer Vision
Oct 19

PanoBench: A Panoptic Occupancy Prediction Benchmark for Autonomous Driving

PanoBench is a new camera‑centric benchmark for panoptic occupancy prediction that unifies 3D geometry, semantics, and instance identities across synthetic and real‑world driving scenes. It combines three datasets—PanoBench‑CarlaOcc, PanoBench‑nuScenes, and PanoBench‑KITTI360—through canonical interfaces and unified evaluation protocols, providing multi‑resolution annotations with persistent instance identities. The benchmark introduces a novel reconstruction pipeline for real‑world data, a unified valid‑mask protocol, and analyses of observation fidelity, geometric completeness, spatial continuity, and temporal consistency, while also benchmarking methods on semantic, panoptic, and 4D occupancy forecasting. whyItMatters":"PanoBench addresses the scarcity of instance‑aware annotations in panoptic occupancy research by offering comprehensive, high‑quality annotations and evaluation protocols that enable more robust development and comparison of autonomous driving perception methods."

By Yi Feng, Zizhan Guo, Junwu E, Yu Ma, Mengtan Zhang, Jin Wu, Wei Ye, Hanli Wang, Qijun Chen, Danwei Wang, Rui Fan
arXiv Computer Vision
Oct 12

Measurable Semantic 3D Gaussian Splatting with Joint Region and Depth Supervision for Interactive Building Inspection

The paper introduces a measurable semantic 3D Gaussian Splatting (3DGS) framework that uses region and depth supervision to improve building inspection from multi‑view imagery. Mask‑based region supervision focuses optimization on target areas, while point‑wise depth supervision aligns Gaussian centers with physical surfaces derived from multi‑view stereo point clouds. The resulting semantic Gaussian field enables component‑level inspection and is deployed in the AEC‑SplatTwin platform, achieving significant reductions in depth error and high semantic segmentation accuracy on UAV‑inspected and smartphone‑captured datasets.

By Yi Zhang, Tian Tan, Ray Kai Leung Su
arXiv Computer Vision
Oct 22

Data-Efficient Semantic Segmentation of 3D Point Clouds via Open-Vocabulary Image Segmentation-based Pseudo-Labeling

The paper introduces PLOVIS, a data‑efficient framework for semantic segmentation of 3D point clouds that tackles three simultaneous data shortages: few training scenes, sparse point‑level labels, and missing 2D image sequences. PLOVIS generates pseudo‑labels by applying an open‑vocabulary image segmentation model directly to 3D point clouds, then refines these labels through a two‑stage filtering process and a class‑balanced memory bank. Experiments on ScanNet, S3DIS, Toronto3D, and Semantic3D show that PLOVIS outperforms existing fine‑tuning and weakly supervised methods under realistic, data‑scarce conditions.

By Takahiko Furuya
arXiv Computer Vision
Sep 18

CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Grounding

CitySTAR introduces a training‑free framework that transforms billion‑scale urban point clouds into a query‑ready scene graph of open‑vocabulary 3D instances, using CodeLLM‑driven tools to supply multimodal evidence for node attributes and spatial relations. It models target‑context topology with paired hypergraphs and performs bidirectional topology verification for structural disambiguation, followed by a Reflective Cross‑modal Grounding module that integrates topology consistency and 2D visual evidence to decide over a metric‑aware 3D context graph. The authors also present CitySTAR‑3D, a benchmark that enhances semantic coverage, instance completeness, bounding‑box fidelity, and spatial‑relation complexity for city‑scale 3D grounding, and report extensive experiments showing consistent improvements in open‑world urban 3D grounding with strong interpretability and generalization.

By Shuai Zhang, Hongye Hou, Qinghe Liu, Zhuoxiao Li, Dongli Wu, Jing Ou, Yuan Liu, Wufan Zhao
arXiv Machine Learning
Oct 23

RaSFNet: Ray-aligned and Scan-aware Fusion Network for LiDAR Semantic Segmentation

RaSFNet introduces a Ray-aligned and Scan-aware Fusion Network for LiDAR semantic segmentation, addressing limitations of projection-based methods by preserving geometric differences within each frustum and modeling spatial relationships across scanning regions. It employs a Ray-Aligned Geometric Distribution Encoding (RAGDE) module, a Scan-Topology-Aware Mamba (STAM) block, and a Range-Constrained Point-Frustum Fusion (RCPFF) mechanism to integrate point cloud geometry, LiDAR scanning characteristics, and range information. The network achieves competitive mIoU scores of 70.2% on SemanticKITTI and 77.7% on nuScenes validation sets.

By Chuang Chen
arXiv Computer Vision
Oct 2

Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers

Lang3DSeg introduces a point‑transformer backbone for open‑vocabulary, annotation‑free 3D LiDAR segmentation, trained from scratch without geometric pre‑training. It tackles noise from 2D‑to‑3D label projections by applying a class‑priority rule and truncating projected instances at depth gaps, thereby correcting depth‑ambiguity errors. The method achieves state‑of‑the‑art results on nuScenes (52.8 % mIoU) and SemanticKITTI (41.4 % mIoU) while operating in real‑time on a single LiDAR sweep.

By Cigdem Kokenoz, Amir Salarpour, Alkim Domeke, Christopher Salas, Pedram MohajerAnsari, Long Cheng, Mert D. Pes\'e, Bing Li
arXiv Computer Vision
Sep 3

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

PointGauss is a 3D-native framework that performs semantic parsing and instance segmentation on 3D Gaussian splatting representations by treating Gaussian primitives as unstructured point sets and extracting scale‑invariant geometric features with Point Transformer V3. It introduces an adaptive region‑of‑interest cropping strategy and an instance‑aware distance‑constrained rasterization pipeline to enable scalable, view‑consistent pixel‑level projections. The authors also release SplatSeg‑360, a cross‑scale benchmark with 32 complex scenes and over 6,300 aligned 2D‑3D masks, and show that PointGauss achieves real‑time performance with state‑of‑the‑art 3D‑mIoU (~90%) and 2D‑mIoU (~80%) scores.

By Wentao Sun, Yiping Chen, John S. Zelek, Jonathan Li