The paper investigates how different 2D image backbones affect indoor semantic occupancy prediction in RGB‑D pipelines. By keeping the projection, depth branch, and occupancy head constant and swapping only the image encoder, the authors find that stronger backbones such as DINOv2 and BLIP2 significantly raise mIoU scores compared to the default ResNet‑50. These results show that the choice of image backbone is a major determinant of 3D occupancy accuracy, outweighing many specialized 3D modules.
By Shizhang Fanga, Wanling Yea, Qi Zheng
arXiv:2608. 07579v1 Announce Type: cross Abstract: The AI City Challenge 2026 Track 1 evaluates multi-camera 3D perception in large indoor warehouses under a synthetic-to-real (Sim2Real) setting; depth is available only for training and validation, so inference is RGB-only.
By Abdullah Naeem, Anav Katwal, Ayon Dey, Noman Khan, Md Tamjidul Hoque
O3N is a novel framework that performs open‑vocabulary occupancy prediction from a single omnidirectional RGB image. It introduces a polar‑spiral voxel embedding (PsM) for continuous 360° spatial representation, an Occupancy Cost Aggregation (OCA) module that unifies geometric and semantic supervision, and a Natural Modality Alignment (NMA) pathway that aligns visual, voxel, and text features. Experiments show state‑of‑the‑art results on QuadOcc and Human360Occ benchmarks, with strong cross‑scene generalization and semantic scalability.
By Mengfei Duan, Hao Shi, Fei Teng, Guoqiang Zhao, Yuheng Zhang, Zhiyong Li, Kailun Yang
VEOcc++ is a voxel‑centric framework that unifies online incremental semantic occupancy mapping with offline global map refinement by maintaining a persistent sparse voxel feature map. During exploration the map expands and integrates local features with reliability‑guided updates, while a context‑aware decoder produces the current semantic occupancy map. After exploration, cached local features are consolidated and combined with the online representation for multi‑scale spatial refinement, achieving state‑of‑the‑art IoU and mIoU scores on EmbodiedOcc‑ScanNet and demonstrating zero‑shot transfer to unseen environments on a mobile robot.
By Ruoyu Wang, Jiahan Li, Yong Liu, Kexin Zhang
The paper investigates how much semantic information is lost when frozen foundation models are combined for few‑shot 3D segmentation. By varying the number of retained semantic alternatives before fusion, the authors show that keeping the full distribution of class scores yields higher harmonic‑mean IoU than collapsing to a single class. Experiments on ScanNet200 and ScanNet++ confirm that full‑distribution fusion consistently outperforms top‑1 and other operators, and that most useful information is recovered by retaining a compact set of plausible alternatives.
By Silas Kwabla Gah, Ebenezer Owusu
Classical image correspondence is solved at the level of sparse keypoints or dense pixels, but the systems that consume these matches - object-level mapping, topological navigation, scene-graph maintenance - reason about whole objects. Recent work narrows this gap by matchng directly at the level of instance segments: a class-agnostic segmenter partitions each image, and per-segment descriptors are obtained by pooling features from large 3D foundation models over the masks.