The paper introduces Out-of-Distribution Semantic Occupancy Prediction, a task that focuses on detecting unknown objects in 3D voxel space for autonomous driving. It proposes Realistic Anomaly Augmentation to create two new datasets, VAA-KITTI and VAA-KITTI-360, and presents the OccOoD framework, which uses Cross‑Space Semantic Refinement to improve OoD detection while maintaining semantic occupancy accuracy. Experiments show OccOoD achieves an AuROC of 65.50% and an AuPRCr of 31.83% within a 1.2 m radius, demonstrating strong generalization to real‑world urban scenes.
By Yuheng Zhang, Mengfei Duan, Kunyu Peng, Yuhang Wang, Ruiping Liu, Fei Teng, Kai Luo, Zhiyong Li, Kailun Yang
CrowdOcc introduces a new RGB‑D dataset and monocular semantic scene completion framework tailored for quadruped robots navigating crowded indoor spaces. The dataset comprises 25.1K frames across 11 scenes, with annotations derived via static‑dynamic decoupling. The framework employs Normal Guided Scene Geometry Fusion and Human‑Centric Sparse Interaction to robustly model occluded geometry and human‑scene relationships, achieving state‑of‑the‑art performance on the dataset’s test set.
By Feiyang Chen, Jincheng Hu, Yiduo Chen, Jihao Li, Yue Liang, Bingzhao Gao, Yanjun Huang, Yuanjian Zhang
arXiv:2610.28530v1 Announce Type: new
Abstract: Collaborative perception enables agents to retrieve the scene semantics and geometry beyond the field of view of any individual agent. This broader sce...
By Om Shivam Verma, Shaya Farahmand, Lingjun Zhao, Lu Gan
Foundation4D is a unified feed‑forward model that performs geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi‑camera driving video. It uses a shared backbone with factorized spatiotemporal attention to merge observations across cameras and time, and a video inference pipeline that refines camera trajectories via pose graph optimization and assembles object tracks. The authors also provide a data annotation pipeline and release the PAI‑Foundation4D benchmark, achieving state‑of‑the‑art results on depth, detection, and occupancy metrics and demonstrating that co‑training detection with geometry and occupancy improves performance.
By NVIDIA, :, S\'ergio Agostinho, Maximilian Apodaca, Emmanuel Attia, Guillem Braso, George Dong, Ilya Fedorov, Michele Fenzi, Gang Huang, Jiahui Huang, Shengyu Huang, Bingxin Ke, Laura Leal-Taix\'e, Kuan-Ting Lee, Mingxuan Liu, Tian Liu, Tim Meinhardt, Aljo\v{s}a O\v{s}ep, Naveen Kumar Rai, Kefei Ren, Cristiano Saltori, Keqiang Wu, Ning Xu, Fei Xue, Jin-Bey Yu, Lei Zhang, Qunjie Zhou
MamMA is a pedestrian trajectory prediction algorithm that leverages LiDAR-generated occupancy maps and egocentric vision sensor data. It partitions the occupancy map into patches to extract obstacle features and incorporates pedestrian awareness states, which influence perception and speed. Using a Mamba-based model, MamMA predicts future trajectories and outperforms state‑of‑the‑art methods on multiple benchmark datasets.
By Juncen Long, Xiaofeng Jin, Gianluca Bardaro, Simone Mentasti, Matteo Matteucci
PanoBench is a new camera‑centric benchmark for panoptic occupancy prediction that unifies 3D geometry, semantics, and instance identities across synthetic and real‑world driving scenes. It combines three datasets—PanoBench‑CarlaOcc, PanoBench‑nuScenes, and PanoBench‑KITTI360—through canonical interfaces and unified evaluation protocols, providing multi‑resolution annotations with persistent instance identities. The benchmark introduces a novel reconstruction pipeline for real‑world data, a unified valid‑mask protocol, and analyses of observation fidelity, geometric completeness, spatial continuity, and temporal consistency, while also benchmarking methods on semantic, panoptic, and 4D occupancy forecasting.
whyItMatters":"PanoBench addresses the scarcity of instance‑aware annotations in panoptic occupancy research by offering comprehensive, high‑quality annotations and evaluation protocols that enable more robust development and comparison of autonomous driving perception methods."
By Yi Feng, Zizhan Guo, Junwu E, Yu Ma, Mengtan Zhang, Jin Wu, Wei Ye, Hanli Wang, Qijun Chen, Danwei Wang, Rui Fan
arXiv:2609.32013v2 Announce Type: replace-cross
Abstract: We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to pri...
By Quinlan Sykora, Sourav Biswas, Christopher Diehl, Andrew Cunningham, Thomas Gilles, Raquel Urtasun
arXiv:2608.13147v2 Announce Type: replace
Abstract: Camera-based autonomous driving perception requires a shared representation that preserves metric 3D structure across synchronized multi-camera str...
By Longfei Xu, Xiaohui Wang, Zehao Huang, Han Li, Ya Yang, Naiyan Wang, Si Liu
OpenBox is a two‑stage automatic annotation pipeline that uses a 2D vision foundation model to associate 2D image cues with 3D point clouds. It then classifies instances by rigidity and motion state to generate adaptive bounding boxes using class‑specific size statistics, eliminating the need for self‑training. Experiments on Waymo Open Dataset, Lyft Level 5 Perception, and nuScenes show improved accuracy and efficiency over existing baselines.
By In-Jae Lee, Mungyeom Kim, Kwonyoung Ryu, Pierre Musacchio, Jaesik Park
Embodied tasks demand accurate, flexible, and semantically rich 3D scene representations. 3D semantic occupancy is well suited to this requirement, as it can model holistic 3D spaces by encoding geome...
iOSPointMapper is a mobile app that performs real‑time, privacy‑conscious sidewalk mapping using on‑device semantic segmentation, LiDAR depth estimation, and fused GPS/IMU data on recent iPhones and iPads. It detects and localizes sidewalk‑relevant features such as traffic signs, traffic lights, and poles, and includes a user‑guided annotation interface for validating outputs before submission. The anonymized data is transmitted to the Transportation Data Exchange Initiative (TDEI), where it integrates with broader multimodal transportation datasets, and evaluations show the app’s potential for enhanced pedestrian mapping.
By Himanshu Naidu, Yuxiang Zhang, Sachin Mehta, Anat Caspi
arXiv:2607. 23384v1 Announce Type: cross Abstract: Data association between landmark measurements and landmark variables has long been a central challenge in SLAM, as estimation accuracy depends critically on associating measurements with the correct landmark variables.
By Yihao Zhang, Jungseok Hong, John J. Leonard