Hugging Face Trending Papers
Jun 4

Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) excel at 2D semantic understanding but lack intrinsic 3D awareness, resulting in representations that fail to maintain geometric and spatial consistency across video frames. Given the scarcity of large-scale 3D data, we present GeoVR, a novel framework that learns geometric representations using purely 2D video sequences.

arXiv Computer Vision
Sep 7

Learning 3D Editing without Paired Supervision via Generative Prior Distillation

The paper introduces a framework for instruction‑guided 3D editing that does not require paired 3D supervision. It distills visual, semantic, and geometric knowledge from foundation models into a 3D editing model using a differentiable rendering pipeline, guided by a 2D visual prior from an image editing model and a semantic prior from a Vision‑Language Model. A 3D‑aware Distribution Matching regularization is added to prevent geometric collapse and ensure realistic 3D outputs, leading to superior instruction fidelity and cross‑view consistency compared to state‑of‑the‑art baselines.

By Hao Wen, Weibin Yun, Hongxing Fan, Haotian Lu, Rui Chen, Zehuan Huang, Lu Sheng
arXiv Computer Vision
Aug 28

DINOcular: Self-Supervised Visuospatial Representations

DINOcular is a self‑supervised framework that learns joint visuospatial representations from RGB‑D observations. It fuses depth‑derived geometric priors with a visual backbone using inter‑patch and intra‑patch fusion, allowing the model to encode both appearance and spatial structure efficiently. The resulting representation improves 3D awareness on multiple geometry benchmarks while staying competitive on standard RGB‑D semantic segmentation tasks.

By Farkhat Almukhamedov, Sami Azirar, Hermann Blum