← Back to all news
arXiv Computer Vision By Hao Vo, Phu Loc Nguyen, Khoa Vo, Sieu Tran, Duc Minh Nguyen, Ngo Xuan Cuong, Nghi D. Q. Bui, Anh Nguyen, Duy Minh Ho Nguyen, Ngan Le

OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs

Read the original on arXiv Computer Vision →

OmniSpace is a plug‑and‑play framework that enhances multimodal large language models (MLLMs) with geometry awareness using only 2D observations. It introduces a Camera Pose Injector, a Multi‑view Epipolar Attention module, and a 3D Geometric Distillation objective to improve cross‑view correspondence and depth estimation. Experiments demonstrate that OmniSpace outperforms existing methods on planning, risk detection, language, and generalization benchmarks for autonomous vehicles.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Oct 21

HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

HERMES++ is a unified driving world model that simultaneously performs 3D scene understanding and future geometry prediction. It uses a BEV representation to merge multi‑view spatial data for LLM compatibility, introduces LLM‑enhanced world queries for knowledge transfer, and employs a Current‑to‑Future Link to condition future geometry on semantic context. A Joint Geometric Optimization strategy enforces structural integrity by combining explicit geometric constraints with implicit latent regularization, and extensive benchmark tests show that HERMES++ outperforms specialist methods in both point cloud prediction and scene understanding.

By Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai
llmsagentsroboticsbenchmarks
More like this →
arXiv AI
Jul 24

3D-Aware VLMs with Implicit and Explicit Geometries

arXiv:2607. 21595v1 Announce Type: cross Abstract: Despite rapid progress, most existing vision-language models (VLMs) built from 2D visual inputs often struggle when handling various 3D tasks that require fine-grained spatial understanding and reasoning.

By Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Ran Xu, Shijian Lu, Gongjie Zhang
llmsmultimodal
More like this →
arXiv Computation and Language
Sep 30

Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering

arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...

By Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon, Federico Tombari, Daniel Barath, Marc Pollefeys, Seungryong Kim, Sunghwan Hong
llmsmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 9

Seek-and-View Reasoning for Multi-View Spatial Understanding

The paper introduces Vantage, a training‑free, model‑agnostic framework that pairs a vision‑language model with a 3D foundation model to perform Seek‑and‑View reasoning for multi‑view spatial understanding. Vantage first analyzes a question to plan a relevant viewpoint, then augments reasoning with geometry‑grounded visual evidence from that viewpoint. Experiments across six VLMs on five benchmarks show consistent improvements without fine‑tuning, demonstrating that revealing spatial evidence through view‑grounded reasoning reduces reliance on language‑based cross‑view alignment.

By Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai, Jingwen Ye
llmsfine-tuningmultimodalbenchmarkssafety
More like this →