arXiv Computer Vision

Think3D: Thinking with Space for Spatial Reasoning

Think3D introduces a framework that endows Vision‑Language Models with interactive 3D chain‑of‑thought reasoning by integrating 3D manipulation tools for active spatial exploration. The approach improves performance on benchmarks such as BLINK Multi‑view, MindCube‑1K, and VSI‑Bench‑Tiny for proprietary models like GPT‑4.1 and Gemini 2.5 Pro, and a reinforcement‑learning variant, Think3D‑RL, enables open‑weight models such as Qwen3‑VL‑4B to autonomously learn effective 3D exploration strategies, yielding tool‑use patterns comparable to stronger models and turning a performance drop on MindCube‑1K into a substantial improvement.

arXiv Computer Vision

Oct 16

Learning to Think and Explore in 3D World

The paper introduces ACT3D, a vision‑language model that learns to think and explore in 3D environments using only single‑image inputs and action‑based interactions. By fine‑tuning with trajectory supervision and then applying reinforcement learning driven by final‑answer correctness, ACT3D develops an exploration policy that adapts to the question and improves accuracy across six benchmarks, outperforming both proprietary and open‑source baselines while using fewer input frames. The study shows that outcome‑based RL can teach VLMs to decide when and how to acquire visual evidence, leading to better reasoning and spatial understanding.

By Yijiang Li, Bingyang Wang, Nuno Vasconcelos

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 12

Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning

arXiv:2606. 12830v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) demonstrate strong multimodal understanding, they remain limited in spatial reasoning tasks that require active evidence acquisition and multi-step visual interaction.

By Changye Li, Meng Lu, Yi Wu, Ligeng Zhu

llms agents multimodal benchmarks

arXiv AI

Jun 12

SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning

arXiv:2606. 13673v1 Announce Type: cross Abstract: Spatial reasoning, the ability to determine where objects are, how they relate, and how they move in 3D, remains a fundamental challenge for vision-language models (VLMs).

By Seokju Cho, Ryo Hachiuma, Abhishek Badki, Hang Su, Byung-Kwan Lee, Chan Hee Song, Sifei Liu, Subhashree Radhakrishnan, Seungryong Kim, Yu-Chiang Frank Wang, Min-Hung Chen

llms agents multimodal benchmarks

arXiv Computation and Language

6d ago

Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?

arXiv:2610.34149v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) excel at zero-shot reasoning, their spatial capabilities are primarily evaluated through static, passi...

By Tianchen Guo, Tengyue Zhang, Chen Liu, Ling Chen, Xin Yu

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 7

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

arXiv:2511. 07403v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning.

By Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin, Cihang Xie, Ronald Clark

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Aug 5

SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

arXiv:2607. 27703v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning.

By Yang Zhou, Zixuan Huang, Sunzhu Li, Zhuo Yang, Chen Zhang, Shunian Chen, Caijun Yan, Jianyao Xu, Shunyu Liu, Weijie Fu, Peiliang Li, Xiaozhi Chen, Yuxiang Cai

llms agents computer-vision reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 17

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

arXiv:2606. 17539v1 Announce Type: cross Abstract: Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging.

By Yatai Ji, An-Chieh Cheng, Yang Fu, Yukang Chen, Han Zhang, Zhaojing Yang, Wei Huang, Ka Chun Cheung, Song Han, Vidya Nariyambut Murali, Pavlo Molchanov, Jan Kautz, Simon See, Hongxu Yin, Ping Luo, Sifei Liu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Computation and Language

Sep 30

Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering

arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...

By Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon, Federico Tombari, Daniel Barath, Marc Pollefeys, Seungryong Kim, Sunghwan Hong

llms multimodal benchmarks

arXiv AI

Jun 2

Distilling Neuro-Symbolic Programs into 3D Multi-modal LLMs

arXiv:2606. 01215v1 Announce Type: cross Abstract: Current 3D spatial reasoning methods face a fundamental trade-off: neuro-symbolic 3D (NS3D) concept learners achieve interpretable reasoning through compositional programs but are constrained to closed-set concept vocabularies and simple programs; end-to-end 3D multi-modal LLMs (3D MLLMs) could handle complex natural language and open-vocabulary concepts but suffer from black-box reasoning without explicit spatial verification.

By Wentao Mo, Yang Liu

llms nlp reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Computer Vision

6d ago

GeoLift: Grounding Video Spatial Reasoning in Explicit 3D Maps

arXiv:2610.33558v1 Announce Type: new Abstract: Video spatial reasoning requires multimodal large language models (MLLMs) to understand how objects are arranged and related across viewpoints. While t...

By Xinglin Li, TingTing Long, Jingzhi Zhou, Jiajing Chen, Jian Yang, Jin Xie

llms multimodal

arXiv Computer Vision

Oct 22

Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models

Think-Capture is a query‑conditioned framework that selectively acquires 3D geometry for task‑critical objects to improve spatial reasoning in Vision‑Language Models. It uses a 3D toolkit to lift zero‑shot masks into a common coordinate frame, a Symbolic 3D State Abstraction to encode object and camera states as structured text, and Hierarchical Group Relative Policy Optimization to train the VLM on when geometry helps or misleads. The resulting Think‑Capture‑4B model achieves significant gains on MindCube‑Tiny, MMSI‑Bench, and ViewSpatial benchmarks while remaining robust to perturbations and fitting on a single 24 GB GPU.

By Junjie Wang, Zhihao Yuan, Yunhan Tang, Sheng Xu, Jiaqi Mao, Xiaoyu Ye, Yueru Luo, Xu Yan, Chun-Mei Feng, Zhen Li

llms multimodal safety

arXiv AI

Jun 2

Limits of Spatial Imagery Reasoning in Frontier LLM Models

arXiv:2603. 26779v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, yet they struggle with spatial tasks that require mental simulation, such as mental rotation.

By Sergio Y. Hayashi, Nina S. T. Hirata

llms

Related stories