AOV-GS is an active 3D Gaussian mapping system that uses frozen foundation models (SAM and CLIP) to guide exploration toward semantically informative regions. During coarse exploration, it re‑ranks geometrically promising views based on CLIP mask‑embedding novelty from SAM instance segmentation, biasing the robot’s trajectory. The system fits a lightweight scene‑specific language field offline to the collected CLIP embeddings, enabling open‑vocabulary text‑query segmentation on novel views and improving language‑field mIoU by 1.3–1.8 percentage points over geometry‑only planning on Replica and Matterport3D.
By Rodion Shkokov, Tatiana Zemskova, Dmitry Yudin
LangMap introduces a human‑verified benchmark for language‑conditioned goal navigation (LGN) that spans four hierarchical semantic levels—scene, room, region, and instance—within real‑world indoor 3D scans. The dataset, built on HM3D, contains 18K tasks with concise and detailed descriptions for 414 object categories, and its contrastive annotation protocol ensures high‑quality, discriminative region and instance labels. Evaluation shows that LangMap’s descriptions improve text‑to‑view matching accuracy by 23 points over GOAT‑Bench and achieve a 92.5% unique‑and‑correct match rate in an independent human audit, while a proposed RGB‑only baseline, PlaNaVid, attains top‑tier success rates without depth or 3D scene representations.
By Bo Miao, Weijia Liu, Jun Luo, Lachlan Shinnick, Jian Liu, Thomas Hamilton-Smith, Yuhe Yang, Zijie Wu, Vanja Videnovic, Feras Dayoub, Anton van den Hengel
arXiv:2609.27076v1 Announce Type: new
Abstract: Open-vocabulary visual grounding enables robots to localise task-relevant entities from natural-language queries without dependence on predefined perce...
By Linus Nwankwo, Muslim Alaran, Christian Rauch, Stanley Chukwuebuka Obilikpa, Elmar Rueckert
ENEAS is a unified, text‑promptable method that simultaneously provides precise instance tracking and high‑quality segmentation, and enables open‑concept discovery of any instance named by a text query. It extends the SeC architecture with a text‑prompting adapter and temporal memory to maintain targets through disappearance and avoid drifting, while a semantic verification layer combines visual embedding matching with conditional VLM refinement to filter ontological errors. Designed for 3D reconstruction, ENEAS delivers robust semantic tracking and segmentation across videos, libraries, and unordered collections, distinguishing true instances from look‑alike doppelgangers.
By Javier del Pino (SperidLabs), Salvador Rodr\'iguez (SperidLabs), Alejandro Garabito (SperidLabs), Javier \'Alvarez (SperidLabs), Chema Garabito (SperidLabs)
GoDeep is an annotation‑free method for open‑vocabulary 3D scene understanding that uses a vision‑language model solely as a translator to generate structured, entity‑level descriptions of each image. These descriptions are projected and aggregated in a language‑only embedding space, eliminating the need for a 3D training corpus or domain‑specific encoder. The approach achieves competitive performance on ScanNet++ and a cultural heritage benchmark, accurately localizes out‑of‑vocabulary objects, and offers explainable, point‑level predictions.
By Thodoris Betsas, Anastasios Doulamis, Andreas Georgopoulos
arXiv:2607. 10383v1 Announce Type: cross Abstract: Visual Language Navigation foundation models aim to unify deep reasoning for grounded spatial decisions with broad versatility for diverse embodied tasks.
By Ruiyan Gong, Yingnan Guo, Junjun Hu, Jintao Kong, Xiaoxu Leng, Tianlun Li, Weize Li, Fei Liu, Zhicheng Liu, Jia Lu, Minghua Luo, Chenlin Ming, Yanfen Shen, Jiyue Tao, Zhengbo Wang, Mingyang Yin, Minqi Gu, Zihao Guan, Wei Guo, Guoqing Liu, Huachong Pang, Menglin Yang, Zeqian Ye, Xiaoxiao Geng, Zhining Gu, Honglin Han, Di Jing, Hongyu Pan, Mingchao Sun, Kuan Yang, Jianfang Zhang, Yanghong Chen, Ye He, Wei Mei, Jiahao Shi, Xiangpo Yang, Yanqing Zhu, Zedong Chu, Xiaolong Wu, Mu Xu