arXiv AI

Uncertainty Gates the Gradient: Adaptive Updates for Generalized Few-Shot 3D Segmentation

arXiv Machine Learning
1d ago

Role-based Effectiveness of an Epistemic Uncertainty Estimate in Training Models for Semantic Segmentation of 3D LiDAR Point Clouds

The paper studies how epistemic uncertainty estimates can be used in different ways—either as an additional input feature or as a per‑sample confidence weight in the loss function—to improve 3D LiDAR point cloud segmentation. Experiments on four large‑scale LiDAR benchmarks (ISPRS Vaihingen, AHN3, Paris‑Lille‑3D, and Toronto‑3D) show that uncertainty‑aware models outperform a baseline ANN in mean Intersection over Union, recover minority classes, and better align prediction error with estimated uncertainty. The results suggest that uncertainty can serve multiple roles in the segmentation pipeline beyond merely boosting accuracy.

By Krishnakumar N, Jaya Sreevalsan-Nair
arXiv Computer Vision
Sep 18

GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model

GAPrompt++ is a multi-granular geometry-aware prompting method designed to adapt pre-trained 3D vision models to downstream tasks efficiently. It introduces a Point Shift Prompter for multi-scale geometric feature extraction, a Keypoint Prompter for local geometric saliency, and a Prompt Propagation mechanism to embed these cues throughout the model hierarchy. Experiments demonstrate that GAPrompt++ outperforms other prompting-based PEFT methods and even surpasses full fine-tuning while using less than 2% trainable parameters, and the authors provide two new challenging benchmarks for future research.

By Zixiang Ai, Zhenyu Cui, Yufei Guo, Wenwen Qiang, Lei Chen, Jiwen Lu, Jiahuan Zhou
arXiv Computer Vision
Sep 18

SenseFuse: Label-Free Fusion of Image and Shape Encoders for Open-Vocabulary 3D Instance Segmentation

SenseFuse introduces a label‑free fusion approach that balances 2D image and 3D shape encoders for open‑vocabulary 3D instance segmentation. By selecting a scene‑level fusion weight through an adaptive, sensitivity‑based mechanism, it improves mask labeling accuracy across multiple datasets, recovering up to 93% of the potential gain from an oracle weight. The method demonstrates that image and shape encoders have complementary failure patterns, leading to higher instance AP in most evaluated settings.

By Euiseok Han, Tri Ton, Hwanhee Kim, Seungyeon Ryu, Chang D. Yoo
arXiv Machine Learning
Sep 23

SAM-V: Geometry-Aware Segment Anything for Multi-View Instance Segmentation

SAM‑V is a geometry‑aware extension of the Segment Anything Model (SAM) that integrates 3D priors from a feed‑forward geometry model (VGGT) into 2D segmentation. It uses a prompt‑fusion mechanism to combine sparse SAM prompts with view‑specific camera tokens and local VGGT features, enabling a mask decoder that attends to both dense 2D and 3D cues. The resulting end‑to‑end system produces consistent multi‑view instance segmentation in a single forward pass, achieving significant gains on the IGGT 3D tracking benchmark without offline mask matching or explicit 3D reconstruction.

By Jiangshan Gong, Yuqun Wu, Qiqian Fu, Yao Xiao, Chuhang Zou, Shenlong Wang, Derek Hoiem
arXiv Computer Vision
Oct 22

Data-Efficient Semantic Segmentation of 3D Point Clouds via Open-Vocabulary Image Segmentation-based Pseudo-Labeling

The paper introduces PLOVIS, a data‑efficient framework for semantic segmentation of 3D point clouds that tackles three simultaneous data shortages: few training scenes, sparse point‑level labels, and missing 2D image sequences. PLOVIS generates pseudo‑labels by applying an open‑vocabulary image segmentation model directly to 3D point clouds, then refines these labels through a two‑stage filtering process and a class‑balanced memory bank. Experiments on ScanNet, S3DIS, Toronto3D, and Semantic3D show that PLOVIS outperforms existing fine‑tuning and weakly supervised methods under realistic, data‑scarce conditions.

By Takahiko Furuya
arXiv AI
Sep 7

Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning

The paper addresses performance discrepancy in cross-domain 3D class‑incremental learning, where 3D point clouds from heterogeneous sources cause varying degrees of performance loss beyond catastrophic forgetting. The authors introduce the Domain3D‑CIL protocol and adapt existing CIL methods to 3D, showing consistent discrepancy across baselines. They propose PolyMem, an exemplar‑free approach that models high‑order feature statistics to improve cross‑domain robustness and reduce performance discrepancy.

By Jinge Ma, Gautham Vinod, Bruce Coburn, Jui-Feng Chi, Siddeshwar Raghavan, Fengqing Zhu