arXiv Computer Vision

TempoGround: State-Aware Streaming Visual Grounding with Vision-Language Models

TempoGround is a vision‑language model–native framework for streaming visual grounding that detects cross‑frame object correspondence and explicitly models object presence states. It uses a curriculum prediction mechanism to resolve 2D instance association, predict object entry, continuation, or exit, decode 2D boxes, and lift them to 3D camera‑frame boxes. The approach is further refined with Streaming Grounding Reinforcement, which optimizes grounding, identity, and consistency rewards, and achieves significant improvements on multiple streaming visual grounding benchmarks.

arXiv AI

Oct 1

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

GroundingPI is a 4‑billion‑parameter grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. It is trained with multimodal and spatial pretraining, supervised fine‑tuning, and reinforcement learning, achieving a new state‑of‑the‑art average of 73.68% across 34 grounding benchmarks. As a visual backbone, GroundingPI improves performance in robotic manipulation and autonomous driving, outperforming larger models and mainstream backbones in several out‑of‑distribution settings.

By Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao Jiang, Wei Wei, Ruihai Wu, Hang Zhang, Yixiao Ge, Shuchang Zhou, Shilong Liu, Xianming Liu, Ping Luo, Shiyu Huang

llms agents computer-vision reinforcement-learning robotics fine-tuning efficiency multimodal benchmarks

arXiv AI

Aug 25

VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting

VisionCoach is an input‑adaptive reinforcement learning framework that enhances spatio‑temporal grounding in video reasoning by using visual prompting during training. The system selectively applies visual prompts to challenging inputs, amplifying question‑relevant evidence and suppressing distractors, and then internalizes these improvements through self‑distillation so that inference can be performed on raw videos without prompts. Experiments on multiple benchmarks (V‑STAR, VideoMME, World‑Sense, VideoMMMU, PerceptionTest, and Charades‑STA) show that VisionCoach achieves state‑of‑the‑art performance while maintaining a single efficient inference pathway.

By Daeun Lee, Shoubin Yu, Yue Zhang, Mohit Bansal

llms reinforcement-learning efficiency benchmarks

arXiv Computer Vision

Oct 12

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

GuidedNav is a new framework for vision‑and‑language navigation that explicitly shapes the representations linking instructions to actions. It uses subtask and landmark guidance to focus pre‑action attention on the relevant instruction segment and visual regions, and spatial guidance to align internal representations with geometric features of the environment. The method achieves state‑of‑the‑art success rates on R2R‑CE and RxR‑CE val‑unseen, improving by 8.1% and 13.1% respectively, while also providing faster inference and introducing two new annotation‑enriched datasets, Guided‑R2R and Guided‑RxR.

By Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong

agents multimodal benchmarks

arXiv AI

Jul 28

The Visual Bottleneck: Sparse-Frame Adaptation of MLLMs for Joint Spatial-Temporal Video Grounding

arXiv:2607. 24570v1 Announce Type: cross Abstract: Large-scale video platforms process millions of uploads hourly, requiring moderation systems that can localize when and where policy violations occur within each video.

By Jiameng Zhang, Srikanth Madikeri

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Aug 25

Traj-VLN: Learning Pixel-Space Interaction via Autoregressive Trajectory Generation

arXiv:2607.10744v5 Announce Type: replace Abstract: Benefiting from the powerful priors embedded in large-scale pre-training data and the emerging commonsense reasoning ability, large language models...

By Changfei Fu, Guangcheng Chen, Aoxiang Gu, Haoxiang Liang, Wenjun Xu, Hong Zhang

llms rag agents robotics fine-tuning multimodal benchmarks

arXiv AI

Jul 14

ABot-N1: Toward a General Visual Language Navigation Foundation Model

arXiv:2607. 10383v1 Announce Type: cross Abstract: Visual Language Navigation foundation models aim to unify deep reasoning for grounded spatial decisions with broad versatility for diverse embodied tasks.

By Ruiyan Gong, Yingnan Guo, Junjun Hu, Jintao Kong, Xiaoxu Leng, Tianlun Li, Weize Li, Fei Liu, Zhicheng Liu, Jia Lu, Minghua Luo, Chenlin Ming, Yanfen Shen, Jiyue Tao, Zhengbo Wang, Mingyang Yin, Minqi Gu, Zihao Guan, Wei Guo, Guoqing Liu, Huachong Pang, Menglin Yang, Zeqian Ye, Xiaoxiao Geng, Zhining Gu, Honglin Han, Di Jing, Hongyu Pan, Mingchao Sun, Kuan Yang, Jianfang Zhang, Yanghong Chen, Ye He, Wei Mei, Jiahao Shi, Xiangpo Yang, Yanqing Zhu, Zedong Chu, Xiaolong Wu, Mu Xu

robotics multimodal benchmarks safety

arXiv AI

Jun 2

Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

arXiv:2604. 17473v3 Announce Type: replace-cross Abstract: Vision-Language Navigation(VLN) requires an agent to navigate through 3D environments by following natural language instructions.

By Kangyi Wu, Pengna Li, Kailin Lyu, Xi Lin, Lin Zhao, Qingrong He, Jinjun Wang, Jianyi Liu

llms rag agents multimodal

arXiv AI

Jul 16

ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding

arXiv:2607. 13421v1 Announce Type: cross Abstract: Spatio-Temporal Video Grounding (STVG) aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression.

By Kai Chen, Ming Dai, Wenxuan Cheng, Wankou Yang

multimodal benchmarks

arXiv AI

Oct 7

GeoPID: Decomposing and Steering Visual Information in Vision-Language Models

arXiv:2610.08401v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) have shown outstanding performance across diverse applications, they tend to under-use visual information...

By Seulgi Kim, Zhixiong Zhang, Xinwei Zhang, Jie Ling, Ronn Shaw

llms multimodal benchmarks

arXiv AI

Oct 2

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...

By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris

llms efficiency multimodal benchmarks

arXiv Computer Vision

Sep 24

Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments

arXiv:2609.27076v1 Announce Type: new Abstract: Open-vocabulary visual grounding enables robots to localise task-relevant entities from natural-language queries without dependence on predefined perce...

By Linus Nwankwo, Muslim Alaran, Christian Rauch, Stanley Chukwuebuka Obilikpa, Elmar Rueckert

robotics efficiency benchmarks

arXiv AI

Jul 3

CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation

arXiv:2607. 02222v1 Announce Type: cross Abstract: Vision-Language Navigation has increasingly emphasized high-level instruction reasoning, memory, global map construction, and instruction decomposition, while the low-level action representation remains comparatively underexplored.

By Haokun Liu, Zhaoqi Ma, Yicheng Chen, Wentao Zhang, Masaki Kitagawa, Zicen Xiong, Jinjie Li, Moju Zhao

robotics multimodal benchmarks

Related stories