← Back to all news
arXiv Computer Vision

Open-vocabulary 3D object detection with promptable segmentation

The paper introduces an open‑vocabulary 3D object detection pipeline that uses a promptable segmentation model (SAM3) to generate instance masks from six surround‑view cameras. These masks are converted into metric 3D boxes, achieving up to 0.413 mAP/0.555 NDS without any training when supervised box geometry is borrowed at inference. The approach also improves a supervised LiDAR‑only detector by 0.034 mAP through a camera‑witness rule, demonstrating that measurement precision, not 2D detection, limits performance.

Related stories

arXiv Computer Vision
Oct 2

Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers

Lang3DSeg introduces a point‑transformer backbone for open‑vocabulary, annotation‑free 3D LiDAR segmentation, trained from scratch without geometric pre‑training. It tackles noise from 2D‑to‑3D label projections by applying a class‑priority rule and truncating projected instances at depth gaps, thereby correcting depth‑ambiguity errors. The method achieves state‑of‑the‑art results on nuScenes (52.8 % mIoU) and SemanticKITTI (41.4 % mIoU) while operating in real‑time on a single LiDAR sweep.

By Cigdem Kokenoz, Amir Salarpour, Alkim Domeke, Christopher Salas, Pedram MohajerAnsari, Long Cheng, Mert D. Pes\'e, Bing Li
llmsagentscomputer-visionmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 22

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

AutoExpert proposes a new benchmark for automating 3D LiDAR annotation using expert‑crafted guidelines, redefining evaluation protocols and re‑annotating nuScenes and PandaSet datasets. The pipeline combines 2D detection and segmentation in RGB images, lifts these detections into 3D with known sensor poses, and generates 3D cuboids, improving 3D detection mAP from 12.1 to 25.4 on the AutoExpert‑nuScenes benchmark.

By Yechi Ma, Wei Hua, Shu Kong
agentscomputer-visionroboticsbenchmarks
More like this →
arXiv AI
1d ago

Occlusion for Free: Detecting Occlusion from the Disagreement Between Two Monocular Distance Cues

The paper demonstrates that occlusion can be inferred from the disagreement between two monocular distance cues—apparent height and metric depth—without requiring any occlusion labels or training. By comparing the ordering of these cues, the authors achieve high occlusion detection performance on KITTI and synthetic CARLA scenes, outperforming distance alone. The method relies only on a camera‑only pipeline and a depth model whose errors are close to a pure gain, a condition that can be verified without labels.

By Chibueze Praise Onyenweaku, Toyibat Adele, Sikirulahi Opeyemi Abdulkareem, Obed Honour Eje
roboticsbenchmarks
More like this →
arXiv Computer Vision
Oct 19

TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue

TriSAR-3D is a new trimodal dataset combining high‑resolution LiDAR, RGB, and thermal imagery to capture people in varied indoor and outdoor poses, including lying and partially occluded positions. The authors propose PaintFusion, a lightweight trimodal 3D detection pipeline that fuses PointPainting evidence with an MLP, achieving 65.4 AP@50 and outperforming LiDAR‑only, RGB+LiDAR, and other trimodal baselines while running faster (51.6 ms per frame). Experiments show that removing lying instances from training drastically reduces detection performance, highlighting the importance of dataset diversity for robust 3D person detection in search and rescue scenarios.

By Markus Kramer, Jan Wachenfeld, Oskar von Stryk
computer-visionroboticsmultimodal
More like this →