arXiv Computer Vision

Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?

Hugging Face Trending Papers

Aug 24

Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?

Vision-language models (VLMs) can answer spatial questions, yet the mechanisms connecting object grounding to spatial reasoning remain poorly understood. It is underexplored whether spatial reasoning...

llms multimodal benchmarks safety

arXiv Machine Learning

Oct 12

S-Space: Exploring Spatial Workspace in Multimodal Language Models

The paper introduces S‑Space, a linear subspace within multimodal language models where object‑token representations encode continuous spatial coordinates along horizontal, vertical, and distance axes. This subspace is intrinsic, readable under non‑spatial queries, and serves as an internal spatial workspace: perception maps object locations into it, reasoning transforms it for viewpoint changes, and verbal responses selectively access it. Diagnostic experiments using S‑Space reveal that while models can form spatial representations, they struggle with spatial transformations needed for reliable reasoning across viewpoint changes.

By Zhiyuan Shi, Xiang Li, Xingzhuo Guo, Shangchen Miao, Weiliang Chen, Qiyu Dai, Jinshan Ren, Jiangran Lyu, Fangfu Liu, Mingsheng Long, Jialong Wu

llms multimodal

arXiv Machine Learning

Sep 10

Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models

arXiv:2609.06880v1 Announce Type: cross Abstract: Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker's situated pers...

By Mimo Shirasaka, Haochen Zhang, Yonatan Bisk

llms robotics multimodal benchmarks

arXiv AI

Jun 16

Thinking with Visual Grounding

arXiv:2606. 16122v1 Announce Type: new Abstract: Visual thinking should not only sound right; it should show its evidence.

By Junkai Zhang, Yihe Deng, Kai-Wei Chang, Wei Wang

llms agents reinforcement-learning multimodal benchmarks

arXiv AI

Oct 1

SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Models

arXiv:2609.38716v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have made remarkable progress across visual perception tasks, yet spatial reasoning remains a persistent weaknes...

By Rafi Ibn Sultan, Xiangyu Zhou, Md. Sajid Alam Chowdhury, Chengyin Li, Prashant Khanduri, Marco Brocanelli, Dongxiao Zhu

llms multimodal benchmarks

arXiv Machine Learning

Jun 19

Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?

arXiv:2605. 20448v2 Announce Type: replace-cross Abstract: Vision-language models reliably name objects in a scene, but do they represent the 3D layout those objects inhabit?

By Animesh Maheshwari, Divyansh Sahu, Nishit Verma

llms robotics multimodal benchmarks

arXiv Machine Learning

Jun 8

The Dual Mechanisms of Spatial Variable Binding in Vision-Language Models

arXiv:2603. 22278v2 Announce Type: replace-cross Abstract: Many multimodal tasks, such as image captioning and visual question answering, require vision-language models (VLMs) to bind objects with their properties and spatial relations.

By Kelly Cui, Nikhil Prakash, Shoval Messica, Ayush Raina, David Bau, Antonio Torralba, Tamar Rott Shaham

llms nlp multimodal

arXiv AI

Jun 3

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

arXiv:2606. 03988v1 Announce Type: new Abstract: Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable.

By Mahtab Bigverdi, Lindsey Li, Weikai Huang, Yiming Liu, Jaemin Cho, Jieyu Zhang, Tuhin Kundu, Chris Dangjoo Kim, Zelun Luo, Linda Shapiro, Ranjay Krishna

llms multimodal benchmarks

arXiv AI

Jun 17

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

arXiv:2606. 17539v1 Announce Type: cross Abstract: Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging.

By Yatai Ji, An-Chieh Cheng, Yang Fu, Yukang Chen, Han Zhang, Zhaojing Yang, Wei Huang, Ka Chun Cheung, Song Han, Vidya Nariyambut Murali, Pavlo Molchanov, Jan Kautz, Simon See, Hongxu Yin, Ping Luo, Sifei Liu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 12

Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning

arXiv:2606. 12830v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) demonstrate strong multimodal understanding, they remain limited in spatial reasoning tasks that require active evidence acquisition and multi-step visual interaction.

By Changye Li, Meng Lu, Yi Wu, Ligeng Zhu

llms agents multimodal benchmarks

arXiv Computation and Language

Aug 25

Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?

arXiv:2608.22916v1 Announce Type: new Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...

By Zeyu Wang, Xinming Xu

llms multimodal

arXiv AI

Aug 5

SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

arXiv:2607. 27703v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning.

By Yang Zhou, Zixuan Huang, Sunzhu Li, Zhuo Yang, Chen Zhang, Shunian Chen, Caijun Yan, Jianyao Xu, Shunyu Liu, Weijie Fu, Peiliang Li, Xiaozhi Chen, Yuxiang Cai

llms agents computer-vision reinforcement-learning robotics fine-tuning multimodal benchmarks

Related stories