arXiv AI

UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception

arXiv Computer Vision

Aug 24

AffordAny: Open-World 3D Affordance Grounding from Monocular RGB Images via Vision-Language-Guided Geometric Reasoning

arXiv:2608.20720v1 Announce Type: new Abstract: Open-world 3D affordance grounding requires localizing functional object parts in 3D given free-form language queries. Existing methods typically assum...

By Junqi Wu, Kaihua Tang, Xuanwen Chen, Hongzhi Li, Jianqiang Huang, Xian-Sheng Hua

llms multimodal benchmarks

arXiv AI

Jul 7

Token-Based Affordance Grounding with Large Vision-Language Models

arXiv:2607. 03595v1 Announce Type: cross Abstract: Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception.

By Seung Il Lee, Qinqian Lei, Daguang Xu, Dong Yang, Robby T. Tan, Yixin Chen, Bo Wang

llms robotics multimodal benchmarks

arXiv Computer Vision

Sep 22

AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation

arXiv:2609.22332v1 Announce Type: cross Abstract: Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act...

By Jiadi You, Qize Yu, Yue Chen, Minghong Cai, Zhide Zhong, Yuran Wang, Bowen Ping, Jiaqi Liang, Zhenhao Shen, Haodong Yan, Yinchuan Li, Ruihai Wu, Xiaojuan Qi, Yingcong Chen

llms diffusion robotics multimodal

arXiv Computer Vision

Aug 24

Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding

arXiv:2608.21136v1 Announce Type: new Abstract: Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supe...

By Jie Xu, Na Zhao

llms agents computer-vision robotics

arXiv Computer Vision

Sep 10

StateVLM: A State-Aware Vision-Language Model for Robotic Affordance Reasoning

arXiv:2605.03927v3 Announce Type: replace Abstract: Vision-language models have demonstrated strong performance across robotic perception and instruction-following tasks. However, they still struggle...

By Xiaowen Sun, Matthias Kerzel, Mengdi Li, Xufeng Zhao, Paul Striker, Stefan Wermter

llms robotics multimodal benchmarks

arXiv Computer Vision

2d ago

XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments

arXiv:2604.18484v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from co...

By Kangan Qian, ChuChu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangjun Ye, Guang Chen, Long Chen, Diange Yang

llms agents reinforcement-learning robotics multimodal benchmarks

arXiv AI

1d ago

MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation

arXiv:2601.09879v2 Announce Type: replace-cross Abstract: Recent progress in medical vision-language models (VLMs) has achieved strong performance on image-level text-centric tasks such as report gen...

By Yang Xing, Jiong Wu, Savas Ozdemir, Ying Zhang, Yang Yang, Wei Shao, Kuang Gong

llms rag computer-vision nlp multimodal benchmarks

arXiv Computer Vision

Aug 26

O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics

O3N is a novel framework that performs open‑vocabulary occupancy prediction from a single omnidirectional RGB image. It introduces a polar‑spiral voxel embedding (PsM) for continuous 360° spatial representation, an Occupancy Cost Aggregation (OCA) module that unifies geometric and semantic supervision, and a Natural Modality Alignment (NMA) pathway that aligns visual, voxel, and text features. Experiments show state‑of‑the‑art results on QuadOcc and Human360Occ benchmarks, with strong cross‑scene generalization and semantic scalability.

By Mengfei Duan, Hao Shi, Fei Teng, Guoqiang Zhao, Yuheng Zhang, Zhiyong Li, Kailun Yang

rag robotics benchmarks safety

arXiv AI

Jun 15

3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding

arXiv:2603. 04976v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its potential in 3D scene understanding remains under-explored.

By Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia, Siyuan Huang

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 1

Geometry-Grounded Unified 3D Perception for Autonomous Driving

arXiv:2608.13147v2 Announce Type: replace Abstract: Camera-based autonomous driving perception requires a shared representation that preserves metric 3D structure across synchronized multi-camera str...

By Longfei Xu, Xiaohui Wang, Zehao Huang, Han Li, Ya Yang, Naiyan Wang, Si Liu

agents computer-vision robotics benchmarks

arXiv AI

Jul 24

3D-Aware VLMs with Implicit and Explicit Geometries

arXiv:2607. 21595v1 Announce Type: cross Abstract: Despite rapid progress, most existing vision-language models (VLMs) built from 2D visual inputs often struggle when handling various 3D tasks that require fine-grained spatial understanding and reasoning.

By Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Ran Xu, Shijian Lu, Gongjie Zhang

llms multimodal

arXiv Computation and Language

3d ago

Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering

arXiv:2609.38177v1 Announce Type: cross Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs...

By Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon, Federico Tombari, Daniel Barath, Marc Pollefeys, Seungryong Kim, Sunghwan Hong

llms multimodal benchmarks

Related stories