Metric-Bench introduces a new benchmark for Vision‑Language Models (VLMs) that focuses on metric‑spatial reasoning in indoor scenes by using in‑image reference objects with known dimensions. The accompanying MetricReasoner fine‑tuning recipe employs structured prompts and numerical rewards to implicitly learn 2D‑to‑3D mapping without camera intrinsics. Experiments show that this approach improves spatial metric understanding by 43.1 % over existing models and boosts downstream embodied tasks, while also delivering gains on general VLM benchmarks.
By Yuling Xi, Haokai Zhang, Muzhi Zhu, Hao Zhong, Zongze Du, Hengyu Zhao, Chenchen Jing, Yufei Yin, Bin Qin, Yongjie Yang, Zhenbo Luo, Hao Chen, Chunhua Shen
AgentVQA is an offline benchmark designed to evaluate vision‑language models on realistic agentic scenarios across five domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. It comprises 1,000 carefully selected real‑world examples drawn from 23 source datasets, enabling reliable comparisons without costly online simulations. The benchmark reveals that current models achieve only 58.5% accuracy, with most errors stemming from difficulties in grounding and tracking evolving visual states rather than from task misunderstanding or reasoning.
By Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu
arXiv:2610.10438v1 Announce Type: new
Abstract: Embodied and assistive agents must do more than recognize objects: they must reason about where an object belongs given the layout of an environment an...
By Xuefei Sun, Lorin Achey, Kali Hamilton, Alberto Speranzon, Gregory Grebe, Yonatan Bisk, Christoffer Heckman
arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.
By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo
SpatialScout is a benchmark that tests vision‑language model agents on active, object‑level 3D mapping. It includes 80 indoor scenes with 3,140 target objects, where agents must control a camera, gather RGB observations, and estimate object centers in a shared coordinate frame. Performance is measured by localization accuracy and target coverage using object‑scale overlap scores, revealing that current VLMs, including Claude Opus 5.5, struggle to build precise metric maps.
By Zhaoxi Zhang, Hao Liang, Linxuan Wu, Yiming Wu, Yifu Yuan, Kun Liang, Chenming Tang, Meiyi Qiang, Wentao Zhang, Weikang Li
arXiv:2603. 09731v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly considered as a foundation for embodied agents, yet it remains unclear whether they can reliably reason about the long-term physical consequences of actions from an egocentric viewpoint.
By Chengjun Yu, Xuhan Zhu, Chaoqun Du, Pengfei Yu, Wei Zhai, Yang Cao, Zheng-Jun Zha
arXiv:2606. 00095v1 Announce Type: cross Abstract: Vision-Language Navigation (VLN) enables embodied agents to reach target locations in unseen environments by following language instructions.
By Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, Liang He
arXiv:2610.25959v2 Announce Type: replace
Abstract: Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general inte...
By X. Feng, J. Tang, Z. Lin, L. Chen, S. Shao, Q. Lu, K. Huang
PhyMix introduces a Physics Evaluator that checks geometric priors, contact, stability, and deployability through nine sub-constraints, establishing a benchmark for physical consistency in single-image 3D indoor scene generation. The framework uses implicit alignment via Scene-GRPO and explicit refinement through verifier-guided agentic repair to incorporate evaluator feedback during training and inference, improving physical plausibility while preserving scene content. Experiments demonstrate enhanced reconstruction quality and physical plausibility, with qualitative results on stylized and real-world images showing the method’s broader applicability.
By Dongli Wu, Jingyu Hu, Ka-Hei Hui, Xiaobao Wei, Chengwen Luo, Jianqiang Li, Zhengzhe Liu
arXiv:2608.30821v1 Announce Type: cross
Abstract: Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and e...
By Minghan Qin, Yuang Wang, Xiuyu Yang, Yushi Long, Yujian Zhang, Ruihuan Wang, Kai Ye, Yangang Zhang, Hang Li
Embodied and assistive agents must do more than recognize objects: they must reason about where an object belongs given the layout of an environment and the habits of the people who live in it. Progre...
CLOSER is an evidence‑aware active visual verification framework for embodied object‑goal navigation. It fine‑tunes a vision‑language model to label each view of a tracked candidate as target, non‑target, or uncertain, then selects complementary views using an embedding‑based diversity gate and fuses their probabilities with a lightweight network. When uncertainty remains, a diffusion policy directs the agent toward the candidate using only the tracked mask and depth until a decisive judgment is reached.
By Pengying Wu, Mingye Jiang, Yunze Hu, Zhejia Wen, Kangjie Zhou, Zongru Yang, Zhe Peng, Chang Liu