← Back to all news
arXiv Computer Vision By Dominik Engel, Yousef Rajeh, Leon Sick, Ivan Viola

Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

Read the original on arXiv Computer Vision →

The paper introduces a query‑time calibration technique for CLIP features lifted into 3D Gaussian reconstructions, enabling open‑vocabulary segmentation via natural‑language queries. By subtracting a shared scene direction, whitening text prototypes, and applying soft clustering with CSLS, the method reduces directional concentration and improves class‑relative feature separation. Across seven lifting methods and four datasets, this calibration consistently boosts mean intersection over union for both multiclass and single‑query segmentation without requiring retraining or re‑encoding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 10

GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

GoDeep is an annotation‑free method for open‑vocabulary 3D scene understanding that uses a vision‑language model solely as a translator to generate structured, entity‑level descriptions of each image. These descriptions are projected and aggregated in a language‑only embedding space, eliminating the need for a 3D training corpus or domain‑specific encoder. The approach achieves competitive performance on ScanNet++ and a cultural heritage benchmark, accurately localizes out‑of‑vocabulary objects, and offers explainable, point‑level predictions.

By Thodoris Betsas, Anastasios Doulamis, Andreas Georgopoulos
llmsragcomputer-visionmultimodalbenchmarks
More like this →