arXiv Computer Vision

Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning with Lightweight Annotations

arXiv AI

Jun 15

3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding

arXiv:2603. 04976v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its potential in 3D scene understanding remains under-explored.

By Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia, Siyuan Huang

llms reinforcement-learning fine-tuning multimodal benchmarks

Hugging Face Trending Papers

Sep 14

SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes

Vision-language models excel at 2D image understanding but remain limited in 3D spatial reasoning. Progress is hindered by limitations in current benchmarks. First, 3D datasets often rely on point clo...

llms multimodal benchmarks

arXiv AI

Sep 16

SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes

arXiv:2609.16233v1 Announce Type: cross Abstract: Vision-language models excel at 2D image understanding but remain limited in 3D spatial reasoning. Progress is hindered by limitations in current ben...

By Anubhav Khanal, Prabigya Acharya, Roshni Poudel, Sujan Kapali, Bigyan Bhatta, Pramish Paudel, Francois Rameau, Danda Pani Paudel

llms multimodal benchmarks

arXiv Computer Vision

Aug 24

AffordAny: Open-World 3D Affordance Grounding from Monocular RGB Images via Vision-Language-Guided Geometric Reasoning

arXiv:2608.20720v1 Announce Type: new Abstract: Open-world 3D affordance grounding requires localizing functional object parts in 3D given free-form language queries. Existing methods typically assum...

By Junqi Wu, Kaihua Tang, Xuanwen Chen, Hongzhi Li, Jianqiang Huang, Xian-Sheng Hua

llms multimodal benchmarks

arXiv AI

Jul 8

OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding

arXiv:2512. 23020v3 Announce Type: replace-cross Abstract: 3D visual grounding aims to locate objects based on natural language descriptions in 3D scenes.

By Wenyuan Huang, Zhenyu Zhang, Zhao Wang, Zhou Wei, Ting Huang, Fang Zhao, Jian Yang

llms benchmarks

arXiv Computation and Language

Sep 30

Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering

arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...

By Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon, Federico Tombari, Daniel Barath, Marc Pollefeys, Seungryong Kim, Sunghwan Hong

llms multimodal benchmarks

arXiv AI

Jul 1

PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding

arXiv:2606. 31148v1 Announce Type: cross Abstract: 3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions.

By Duc Cao Dinh, Khai Le-Duc, Florent Draye, Chris Ngo, Terry Jingchen Zhang, Bernhard Sch\"olkopf, Zhijing Jin

llms efficiency multimodal benchmarks

arXiv Computer Vision

3d ago

P$^3$-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models

arXiv:2610.28018v1 Announce Type: new Abstract: Examining existing benchmarks for grounding in 3D VLMs reveals a pervasive "semantic leakage" in the evaluation and prompting protocols: a heavy relian...

By Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey, Luc Van Gool, Danda Pani Paudel

llms multimodal benchmarks

arXiv Computer Vision

Aug 24

Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding

arXiv:2608.21136v1 Announce Type: new Abstract: Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supe...

By Jie Xu, Na Zhao

llms agents computer-vision robotics

arXiv AI

Sep 10

MV-STRIDE: Enabling MLLMs to Master Multi-View Spatial Reasoning via Hierarchical Capability Modeling

MV-STRIDE is a Multi‑View hierarchical Spatial Reasoning dataset that models dependencies among perception, scene understanding, and contextual reasoning to support 3D spatial cognition. It introduces a QA generation pipeline that enforces cross‑view constraints, producing multi‑level reasoning tasks with chain‑of‑thought supervision. Experiments show that training on MV‑STRIDE yields state‑of‑the‑art performance on multi‑view spatial benchmarks, enabling MLLMs to reason robustly across diverse viewpoints.

By Jin Xu, Xiaojian Huang, Zhuodong Luo, Zhihong Zhang, Xin Liu, Jiansheng Wei, Xinzhi Wang, Jie Zhao, Xuejin Chen

llms multimodal benchmarks

arXiv AI

Jun 17

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

arXiv:2606. 17539v1 Announce Type: cross Abstract: Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging.

By Yatai Ji, An-Chieh Cheng, Yang Fu, Yukang Chen, Han Zhang, Zhaojing Yang, Wei Huang, Ka Chun Cheung, Song Han, Vidya Nariyambut Murali, Pavlo Molchanov, Jan Kautz, Simon See, Hongxu Yin, Ping Luo, Sifei Liu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Aug 13

SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward

arXiv:2608. 12220v1 Announce Type: cross Abstract: Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning.

By Zile Zhou, Huining Yuan, Weichen Zhang, Xinlei Chen, Xiao-ping Zhang

llms reinforcement-learning multimodal benchmarks

Related stories