VABench is a benchmark that tests general‑purpose multimodal large language models (MLLMs) on embodied spatial intelligence by requiring them to observe, reason, act, and revise based on visual demonstrations and active perception. The benchmark includes 14 task families, a fixed model‑agnostic controller, and evaluates models on target localization, spatial relations, and long‑horizon composition tracks without providing privileged object poses or learned action heads. Results show that while the best model achieves perfect target localization, overall task success remains modest, and active camera control and geometric transfer significantly influence performance.
By Zhongbo Zhang, Jiayi Jin, Yifan Wang, Zaibin Zhang, Haiwen Diao, Lijun Wang, Huchuan Lu
SceneTeract is a verification interface that separates semantic action understanding from physical feasibility in indoor 3D scenes. It decomposes activities into atomic actions and performs explicit geometric checks to determine executability, providing diagnostic traces for failures. The system reveals widespread functional and accessibility issues in synthetic scenes, shows that existing VLMs over‑predict action feasibility, and improves VLM performance through post‑training with verifier feedback, with benefits that generalize to real‑world scenes.
By L\'eopold Maillard, Francis Engelmann, Tom Durand, Boxiao Pan, Yang You, Leonidas Guibas, Maks Ovsjanikov
arXiv:2609.36906v1 Announce Type: new
Abstract: Reliable embodied decisions under partial observability require informative observations and sufficient supporting evidence. However, semantic scores a...
By Sean Hardesty Lewis, Zuyi Guo, Benwang Chen, Zirui Liu, Hongyi Lin, Heye Huang
arXiv:2606. 14772v1 Announce Type: cross Abstract: Aerial Embodied Question Answering (EQA) requires Unmanned Aerial Vehicles (UAVs) to actively perceive the environment and answer natural language questions.
By Wenhao Lu, Zhengqiu Zhu, Xiaofeng Wang, Xiaoran Zhang, Yatai Ji, Yong Zhao, Yue Hu, Yingzhen Nie, Jinlong Zhu, Zheng Zhu
VeriFuse is a bounded arbitration framework that integrates vision‑language models (VLMs) into vehicle‑infrastructure cooperative 3D perception. Each agent first generates independent detections, then VeriFuse creates a unified candidate pool of geometric proposals and cross‑source hypotheses. A frozen VLM selects among three actions—SELECT, REFINE, or REJECT—to resolve ambiguity and produce final 3D detections, achieving strong AP50/AP70 scores on the DAIR‑V2X dataset while keeping vehicle‑side BEV AP50 drop minimal under delay.
By Hongyi Lin, Yiyao Liu, Qi Kang, Heye Huang, Yang Liu, Haris Koutsopoulos, Jinhua Zhao
The paper introduces ACT3D, a vision‑language model that learns to think and explore in 3D environments using only single‑image inputs and action‑based interactions. By fine‑tuning with trajectory supervision and then applying reinforcement learning driven by final‑answer correctness, ACT3D develops an exploration policy that adapts to the question and improves accuracy across six benchmarks, outperforming both proprietary and open‑source baselines while using fewer input frames. The study shows that outcome‑based RL can teach VLMs to decide when and how to acquire visual evidence, leading to better reasoning and spatial understanding.
By Yijiang Li, Bingyang Wang, Nuno Vasconcelos
SeekVLN is a new framework for Vision‑Language Navigation that addresses the problem of agents acting on insufficient evidence, termed Progress Myopia. It combines semantic progress reasoning with active evidence seeking, trained first with Future‑guided Reverse Generation to augment expert trajectories, and then refined via Counterfactual Contrastive Policy Optimization to reward beneficial seeking actions. Experiments on simulated benchmarks show significant gains, improving success rates by 12.7% on R2R‑CE and 7.5% on RxR‑CE, and real‑world tests demonstrate human‑like evidence‑seeking behavior.
By Zhimin Wang, Meiyuan Zhu, Duo Wu, Linjia Kang, Yajun Wang, Yuan Ni, Xiaohang Wang, Tianlu Pan, Jingyan Jiang, Yaowei Wang, Zhi Wang
arXiv:2609.18058v1 Announce Type: new
Abstract: Finding the object referred to by language in a partially observed 3D scene is a core capability for embodied agents. Existing approaches either couple...
By Shixiong Xu, Zhiyuan Chen, Song Ding, Rui Luo, Xiaowei Liang, Dongxu Miao, Zhiying Du
arXiv:2607. 21400v1 Announce Type: cross Abstract: Vision-and-Language Navigation (VLN) enables embodied agents to follow natural-language instructions.
By Jiabin Lou, Haopeng Wang, Yuanshuai Wang, Xinyu Liu, Xuxin Lv, Yuxin Guo, Lei Huang, Rongye Shi, Wenjun Wu
The paper introduces DroneCATS-Agent, a modular framework that places a multimodal large language model (MLLM) at the core of a drone’s control loop, allowing the model to decide actions solely from prompts. It presents the DroneCATS benchmark, evaluating MLLMs on four tasks—approaching, tracking, searching, and multi‑drone commanding—without fine‑tuning or function‑calling. Results show that while small open models can navigate reliably, they often fail by mismanaging protocol termination, highlighting a gap between perception and action planning in current MLLMs.
By Jaewoo Park, Minyoung Lee, Sukmin Seo, Moonbin Yim, Hyunwook Yoon, Dohoon Ryu, Daehee Kim, Myungseo Song, Jihyuk Byun, Seunggyu Chang, Taeho Kil, Jiseob Kim, Bado Lee, Geewook Kim
arXiv:2609.09396v1 Announce Type: new
Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...
By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali
SpatialScout is a benchmark that tests vision‑language model agents on active, object‑level 3D mapping. It includes 80 indoor scenes with 3,140 target objects, where agents must control a camera, gather RGB observations, and estimate object centers in a shared coordinate frame. Performance is measured by localization accuracy and target coverage using object‑scale overlap scores, revealing that current VLMs, including Claude Opus 5.5, struggle to build precise metric maps.
By Zhaoxi Zhang, Hao Liang, Linxuan Wu, Yiming Wu, Yifu Yuan, Kun Liang, Chenming Tang, Meiyi Qiang, Wentao Zhang, Weikang Li