arXiv AI

ExtrinSplat: Decoupling Geometry and Semantics for Open-Vocabulary Understanding in 3D Gaussian Splatting

ExtrinSplat is a new framework that separates geometry from semantics in 3D Gaussian Splatting scenes. It clusters Gaussians into overlapping 3D object groups and uses a Vision‑Language Model to generate lightweight textual hypotheses, creating an extrinsic index layer that handles complex polysemy. This approach reduces adaptation time from hours to minutes, cuts storage overhead by orders of magnitude, and outperforms existing embedding‑based methods on open‑vocabulary 3D object selection and semantic segmentation benchmarks.

Hugging Face Trending Papers

Jul 30

MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians

Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive memory overhead of storing dense language features.

rag computer-vision nlp benchmarks

arXiv AI

Jun 19

QueryGaussian: Scalable and Training-Free Open-Vocabulary 3D Instance Retrieval

arXiv:2606. 19733v1 Announce Type: cross Abstract: Efficiently retrieving specific 3D instances from large-scale scenes via natural language prompts remains a formidable challenge in multimedia analysis.

By Xiuyuan Zhu, Ke Lu, Zijie Yang, Chao Yue, Jian Xue, Dongming Zhang

rag computer-vision efficiency benchmarks

arXiv AI

Aug 13

CORE-3D: Context-aware Open-vocabulary Retrieval by Embeddings in 3D

arXiv:2509. 24528v4 Announce Type: replace-cross Abstract: Object retrieval from a scene has become a new trend of research due to its numerous applications.

By Mohamad Amin Mirzaei, Pantea Amoie, Ali Ekhterachian, Matin Mirzababaei, Babak Khalaj

llms rag computer-vision multimodal benchmarks

Hugging Face Trending Papers

Jul 21

ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting

Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries.

llms computer-vision multimodal benchmarks

Hugging Face Trending Papers

Jul 7

GaussFusion: Towards Multimodal 3D Gaussian Pretraining

3D Gaussian Splatting provides an explicit representation that jointly models geometry and appearance, serving as a scalable foundation for 3D representation learning. Existing pre-training methods for Gaussian representations, such as masked Gaussian reconstruction, primarily capture local structures but offer limited semantic supervision.

multimodal safety

Hugging Face Trending Papers

Aug 17

GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generation

Driving World Models (DWMs) have recently advanced rapidly with generative models, yet most existing methods mainly focus on conditional scene generation and lack explicit 3D scene understanding, lang...

nlp robotics multimodal benchmarks safety

arXiv Machine Learning

Aug 4

3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering

arXiv:2608. 01185v1 Announce Type: cross Abstract: Recent 3D vision-language models (3D VLMs) construct geometry aware tokens by projecting 2D visual features into world coordinates, enabling spatial reasoning for tasks such as 3D question answering.

By Changwoo Baek, Kyeongbo Kong

llms nlp efficiency multimodal benchmarks

arXiv Computer Vision

1d ago

Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalization

arXiv:2608.22740v1 Announce Type: new Abstract: Generalizable 3D Gaussian Splatting (G-3DGS) has emerged as a promising approach for novel view synthesis undersparse-view settings. However, existing...

By Zeyang Bai, Yunpeng Wang, Yunbiao Wang, Jun Xiao

llms rag benchmarks

arXiv Machine Learning

Jun 24

Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints

arXiv:2606. 24353v1 Announce Type: cross Abstract: Bird's-eye view (BEV) perception fuses multi-camera images into a unified top-down representation for autonomous driving.

By Hojun Choi, Seulbin Hwang, Dae Jung Kim, Kisung Kim, Hyunjung Shim, Jinhan Lee

llms agents computer-vision robotics efficiency multimodal benchmarks

arXiv Computer Vision

2d ago

Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding

arXiv:2608.21136v1 Announce Type: new Abstract: Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supe...

By Jie Xu, Na Zhao

llms agents computer-vision robotics

arXiv AI

Jul 7

Generative Semantic Multi-Object Tracking: A Large-Scale Benchmark and an MLLM-Driven Reasoning Framework

arXiv:2601. 06550v3 Announce Type: replace-cross Abstract: Semantic Multi-Object Tracking (SMOT) is evolving from purely geometric localization toward comprehensive video understanding.

By Pan Liao, Feng Yang, Di Wu, Jinwen Yu, Wang Zhao, Dingwen Zhang

llms multimodal benchmarks safety

arXiv AI

Jul 1

PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding

arXiv:2606. 31148v1 Announce Type: cross Abstract: 3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions.

By Duc Cao Dinh, Khai Le-Duc, Florent Draye, Chris Ngo, Terry Jingchen Zhang, Bernhard Sch\"olkopf, Zhijing Jin

llms efficiency multimodal benchmarks

Related stories