The paper introduces DeSpatial, a framework that separates spatial understanding into perception and reasoning. It uses a shared 3D perception tool with joint segmentation–depth co-refinement and cross-view instance fusion to produce a unified, metric, gravity-aligned 3D scene. DeSpatial achieves state‑of‑the‑art results on VSI‑Bench, MindCube‑Tiny, ViewSpatial, and VSI‑SUPER, demonstrating that the main bottleneck in spatial reasoning is precise 3D perception.
arXiv:2610.07982v1 Announce Type: new Abstract: Monocular metric depth estimation and 3D visual grounding represent the two complementary cornerstones of monocular 3D spatial understanding (M3Sun), f...
Think3D introduces a framework that endows Vision‑Language Models with interactive 3D chain‑of‑thought reasoning by integrating 3D manipulation tools for active spatial exploration. The approach improves performance on benchmarks such as BLINK Multi‑view, MindCube‑1K, and VSI‑Bench‑Tiny for proprietary models like GPT‑4.1 and Gemini 2.5 Pro, and a reinforcement‑learning variant, Think3D‑RL, enables open‑weight models such as Qwen3‑VL‑4B to autonomously learn effective 3D exploration strategies, yielding tool‑use patterns comparable to stronger models and turning a performance drop on MindCube‑1K into a substantial improvement.
arXiv:2607. 27703v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning.
arXiv:2606. 13673v1 Announce Type: cross Abstract: Spatial reasoning, the ability to determine where objects are, how they relate, and how they move in 3D, remains a fundamental challenge for vision-language models (VLMs).
arXiv:2606. 19253v1 Announce Type: cross Abstract: Existing approaches to 3D scene understanding in Vision-Language Models (VLMs) either rely on complex, model-specific geometry encoders or large training budgets in pursuit of spatial reasoning.
As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong reasoning but often struggle with fine-grained spatial understanding and object hallucination.
arXiv:2606.22694v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) remain unreliable when spatial reasoning requires composing relations whose meanings depend on frames of reference. E...
OmniAct3D is a framework that adapts Vision Foundation Model (VFM) detectors trained on perspective images to equirectangular projection (ERP) panoramic images for 3D detection. It introduces the ERP-Ray Geometry Adapter (ERGA-Ray) to model spherical rays, the Visual-Action Reasoning Chain (VARC) to ground hypotheses in panoramic evidence, and the Appearance-Guided Heading Expert (AGHE) to recover high‑resolution heading cues. The method achieves a 2.96 NDS improvement on Spheriverse and a 24.87 mAP gain on PanoMMOcc, while preserving most of the baseline’s performance across configurations.
arXiv:2607. 08970v1 Announce Type: cross Abstract: Recent benchmarks for VLMs largely assess single- or limited-view perception, leaving untested the core cognitive ability to integrate observations across viewpoints into a coherent, world-centric (allocentric) 3D mental model.
arXiv:2608. 05242v1 Announce Type: new Abstract: In this work, we explore an alternative paradigm for spatial reasoning by explicitly disentangling 3D perception from reasoning, rather than jointly acquiring implicit 3D perception and reasoning through large-scale training.
arXiv:2606. 12830v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) demonstrate strong multimodal understanding, they remain limited in spatial reasoning tasks that require active evidence acquisition and multi-step visual interaction.
Think-Capture is a query‑conditioned framework that selectively acquires 3D geometry for task‑critical objects to improve spatial reasoning in Vision‑Language Models. It uses a 3D toolkit to lift zero‑shot masks into a common coordinate frame, a Symbolic 3D State Abstraction to encode object and camera states as structured text, and Hierarchical Group Relative Policy Optimization to train the VLM on when geometry helps or misleads. The resulting Think‑Capture‑4B model achieves significant gains on MindCube‑Tiny, MMSI‑Bench, and ViewSpatial benchmarks while remaining robust to perturbations and fitting on a single 24 GB GPU.