LetOccVote is a weakly supervised framework for 3D occupancy prediction that uses 2D pseudo-labels from vision foundation models. It introduces cross‑frame voting mechanisms—Depth Vote for geometric consistency and Semantic Vote for semantic reliability—to filter out erroneous pseudo-labels before training. The method achieves state‑of‑the‑art results on Occ3D‑nuScenes using only 2D supervision.
arXiv:2610.28530v1 Announce Type: new Abstract: Collaborative perception enables agents to retrieve the scene semantics and geometry beyond the field of view of any individual agent. This broader sce...
SplatLabel is an automated pipeline that uses a 4D Gaussian representation to generate LiDAR segmentation and semantic occupancy grids with predictive confidence. It models dynamic scenes through an explicit temporal manifold, tracking moving actors without requiring pre‑annotated 3D bounding boxes. By integrating 360‑degree LiDAR depth maps and distilling soft probabilities from 2D models, it resolves semantic ambiguities over time and space, and evaluates pseudo‑labels via a selective classification framework that balances precision and recall.
GaussianDS introduces a depth‑supervised framework for 3D Gaussian Splatting that jointly optimizes RGB appearance, depth, and compact semantics from scratch. By arranging multi‑view images into a pose‑aware pseudo‑video and propagating view‑consistent masks via SAM2, the method aligns semantic lifting with geometric cues, using depth supervision and edge‑aware refinement to curb semantic drift and boundary leakage. The approach achieves state‑of‑the‑art performance on LERF and 3D‑OVS benchmarks while preserving high‑fidelity reconstruction and enabling downstream tasks such as 3D object removal.
Lang3DSeg introduces a point‑transformer backbone for open‑vocabulary, annotation‑free 3D LiDAR segmentation, trained from scratch without geometric pre‑training. It tackles noise from 2D‑to‑3D label projections by applying a class‑priority rule and truncating projected instances at depth gaps, thereby correcting depth‑ambiguity errors. The method achieves state‑of‑the‑art results on nuScenes (52.8 % mIoU) and SemanticKITTI (41.4 % mIoU) while operating in real‑time on a single LiDAR sweep.
Hybrid Gaussians is a new 3D representation that jointly models object association and language-aligned semantics for open‑vocabulary 3D segmentation. It uses a Multi‑View Object Association mechanism that combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination, and a Boundary Reconstruction Optimization to refine local boundary structure. Experiments on LERF and 3D‑OVS show strong quantitative and qualitative results, achieving 59.1% mIoU on LERF, a 13.4% relative gain over the baseline.
arXiv:2610.00040v1 Announce Type: new Abstract: Recent advances in 3D Gaussian Splatting have enabled open-vocabulary and referring segmentation by distilling semantic knowledge from 2D foundation mo...
arXiv:2609.18898v1 Announce Type: new Abstract: Training-free weighted aggregation is widely used to lift 2D semantic features onto 3D Gaussians for open-vocabulary scene understanding, yet its theor...
PanoBench is a new camera‑centric benchmark for panoptic occupancy prediction that unifies 3D geometry, semantics, and instance identities across synthetic and real‑world driving scenes. It combines three datasets—PanoBench‑CarlaOcc, PanoBench‑nuScenes, and PanoBench‑KITTI360—through canonical interfaces and unified evaluation protocols, providing multi‑resolution annotations with persistent instance identities. The benchmark introduces a novel reconstruction pipeline for real‑world data, a unified valid‑mask protocol, and analyses of observation fidelity, geometric completeness, spatial continuity, and temporal consistency, while also benchmarking methods on semantic, panoptic, and 4D occupancy forecasting. whyItMatters":"PanoBench addresses the scarcity of instance‑aware annotations in panoptic occupancy research by offering comprehensive, high‑quality annotations and evaluation protocols that enable more robust development and comparison of autonomous driving perception methods."
VoxelFix is a graph‑based post‑hoc semantic correction method that refines voxel labels in completed 3D voxel maps while preserving their geometry and occupancy. It learns to correct errors by exploiting local geometry and neighboring semantic information, using training pairs generated by corrupting annotated maps with class confusions from upstream perception pipelines. Experiments on OccuFly maps show consistent improvements of 4.23–5.00 percentage points in mIoU, especially for tree, roof, and wall classes, and the method generalizes to out‑of‑distribution aerial scenes.
arXiv:2608.21136v1 Announce Type: new Abstract: Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supe...
arXiv:2604.11007v2 Announce Type: replace Abstract: Semantic segmentation of 3D point cloud scenes is a crucial task for various applications. In real-world scenarios, training segmentation models of...
Foundation4D is a unified feed‑forward model that performs geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi‑camera driving video. It uses a shared backbone with factorized spatiotemporal attention to merge observations across cameras and time, and a video inference pipeline that refines camera trajectories via pose graph optimization and assembles object tracks. The authors also provide a data annotation pipeline and release the PAI‑Foundation4D benchmark, achieving state‑of‑the‑art results on depth, detection, and occupancy metrics and demonstrating that co‑training detection with geometry and occupancy improves performance.