From Alignment to Fusion in 3D Vision-Language
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The paper introduces Multi-View Relational Distillation (MVRD), a method that distills patch-wise cosine similarities across multiple views from geometry‑grounded vision models into vision‑language models (VLMs). By focusing on relational geometry rather than raw teacher features, MVRD preserves the pretrained alignment between visual and textual representations while enhancing spatial reasoning. Experiments show that MVRD outperforms supervised fine‑tuning and feature distillation, approaching feature‑fusion performance with fewer parameters and lower latency, and it generalizes to 3D scene understanding tasks such as object grounding, dense captioning, and question answering.
arXiv:2608.30451v1 Announce Type: new Abstract: Image-based 3D visual grounding is critical for embodied agents, yet existing benchmarks suffer from loose text-observation alignment and neglect tempo...
arXiv:2606. 31148v1 Announce Type: cross Abstract: 3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions.
Image-based 3D visual grounding is critical for embodied agents, yet existing benchmarks suffer from loose text-observation alignment and neglect temporal ordering. We introduce SeqAlign3DVG, a novel...
3D-SceneQ is a new 3D large language model that tackles two key limitations of existing 3D scene understanding pipelines: the inclusion of irrelevant object tokens that add noise, and the exclusive use of point cloud data that lacks rich semantic cues. It introduces a Query‑Guided Adaptive Pruning (QGAP) module that selectively keeps task‑relevant objects based on user instructions, and a Multi‑modal Object‑level Feature Enhancement (MOFE) module that fuses 2D feature embeddings into the 3D representation to enrich semantic information. Experiments show that 3D‑SceneQ surpasses state‑of‑the‑art methods on several benchmarks, achieving significant gains in metrics such as CIDEr and EM@1.
arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...