arXiv Computation and Language
Sep 30

Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering

arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...

By Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon, Federico Tombari, Daniel Barath, Marc Pollefeys, Seungryong Kim, Sunghwan Hong
arXiv AI
Sep 10

Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning

The paper introduces TTL‑SR, a geometry‑aware Test‑Time Learning framework designed to improve quantitative spatial reasoning in visual‑language models. By augmenting queries with geometrically coupled auxiliary prompts, filtering unreliable predictions, and updating models with a geometry‑aware multi‑objective loss on unlabeled test data, TTL‑SR adapts models to new domains without additional 3D supervision. Experiments show substantial accuracy gains on the Q‑Spatial‑ScanNet dataset for two state‑of‑the‑art VLMs.

By Gege Zhang, Shuaicheng Niu, Gang Dai, Lei Sun, Shuangping Huang
arXiv Computer Vision
Oct 9

Seek-and-View Reasoning for Multi-View Spatial Understanding

The paper introduces Vantage, a training‑free, model‑agnostic framework that pairs a vision‑language model with a 3D foundation model to perform Seek‑and‑View reasoning for multi‑view spatial understanding. Vantage first analyzes a question to plan a relevant viewpoint, then augments reasoning with geometry‑grounded visual evidence from that viewpoint. Experiments across six VLMs on five benchmarks show consistent improvements without fine‑tuning, demonstrating that revealing spatial evidence through view‑grounded reasoning reduces reliance on language‑based cross‑view alignment.

By Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai, Jingwen Ye
arXiv AI
Oct 2

GeoLatent: Geometry-Guided Latent Structuring with Routed Optimization for 3D Reasoning

GeoLatent introduces a geometry‑guided latent structuring approach for 3D reasoning from 2D images, separating position, direction, and global geometry under geometric supervision. It combines Common–Residual Geometry Alignment (CR‑GEO) with routed optimization to prevent latent collapse and to direct visual answer learning through the latents while maintaining full attention. The method achieves state‑of‑the‑art performance on SPAR‑Bench and SPBench, improving geometry effective rank and overall accuracy.

By Yakun Zhu, Yi Bin, Yujuan Ding, Zheng Wang, Pengpeng Zeng, Duo Peng, Jingkuan Song, Heng Tao Shen
arXiv Computer Vision
4d ago

DK-VLM: Geometry as Position for Spatial Reasoning in Vision-Language Models

DK‑VLM introduces Dual‑Kernel Attention, where a visual token’s attention is computed from both its input position and its geometric position derived from 3D points and camera poses. A semantic kernel preserves pretrained attention while a geometric kernel modulates attention via point displacements and relative view transformations, with a learned gate fusing the two. The model also employs Reference‑Frame Composition supervision to update its reference frame across successive judgments, achieving top scores on six spatial benchmarks and strong performance on VSI‑Bench and ReVSI‑32.

By Han Su, Meng Tian, Tianyu Huang, Mingyu Gao, Lennox Wei, Dongxu Li, Jianhua Han, Hang Xu, Xiaohe Wu, Wangmeng Zuo