arXiv Computer Vision

Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds

arXiv Computer Vision
Sep 3

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

PointGauss is a 3D-native framework that performs semantic parsing and instance segmentation on 3D Gaussian splatting representations by treating Gaussian primitives as unstructured point sets and extracting scale‑invariant geometric features with Point Transformer V3. It introduces an adaptive region‑of‑interest cropping strategy and an instance‑aware distance‑constrained rasterization pipeline to enable scalable, view‑consistent pixel‑level projections. The authors also release SplatSeg‑360, a cross‑scale benchmark with 32 complex scenes and over 6,300 aligned 2D‑3D masks, and show that PointGauss achieves real‑time performance with state‑of‑the‑art 3D‑mIoU (~90%) and 2D‑mIoU (~80%) scores.

By Wentao Sun, Yiping Chen, John S. Zelek, Jonathan Li
arXiv Computer Vision
Sep 18

GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model

GAPrompt++ is a multi-granular geometry-aware prompting method designed to adapt pre-trained 3D vision models to downstream tasks efficiently. It introduces a Point Shift Prompter for multi-scale geometric feature extraction, a Keypoint Prompter for local geometric saliency, and a Prompt Propagation mechanism to embed these cues throughout the model hierarchy. Experiments demonstrate that GAPrompt++ outperforms other prompting-based PEFT methods and even surpasses full fine-tuning while using less than 2% trainable parameters, and the authors provide two new challenging benchmarks for future research.

By Zixiang Ai, Zhenyu Cui, Yufei Guo, Wenwen Qiang, Lei Chen, Jiwen Lu, Jiahuan Zhou
arXiv AI
Oct 16

SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning

The paper identifies a theoretical incompatibility between 3D point cloud masked autoencoding and SE(3)-equivariant networks, showing that standard masking causes a topological collapse that forces models to hallucinate orientation. To address this, the authors introduce SIEVE, an SE(3)-equivariant self-supervised framework that compresses point cloud geometry into invariant scalar embeddings using a spherical projection mask that preserves global pose while obfuscating local semantics. Experiments demonstrate that SIEVE avoids the predicted collapse where constant and noise masks fail, and its embeddings outperform coordinate-based baselines and hand-crafted geometric descriptors on downstream point cloud tasks, especially in generalization to unseen shape categories.

By Jongann Lee, Sun Woo Park, Yun Young Choi