arXiv AI

Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation

SeekVLN is a new framework for Vision‑Language Navigation that addresses the problem of agents acting on insufficient evidence, termed Progress Myopia. It combines semantic progress reasoning with active evidence seeking, trained first with Future‑guided Reverse Generation to augment expert trajectories, and then refined via Counterfactual Contrastive Policy Optimization to reward beneficial seeking actions. Experiments on simulated benchmarks show significant gains, improving success rates by 12.7% on R2R‑CE and 7.5% on RxR‑CE, and real‑world tests demonstrate human‑like evidence‑seeking behavior.

Hugging Face Trending Papers
Jul 2

Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigation

On-policy exploration is a crucial component for training robust Vision-Language Navigation agents, as it exposes the policy to a broader state distribution. However, such exploration inevitably leads to trajectories that deviate from expert demonstrations, resulting in a semantic mismatch between the executed visual stream and the original language instruction.

arXiv AI
Oct 9

More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models

The paper introduces DriveMind, a large-scale driving Visual Question Answering corpus that aligns chain-of-thought reasoning with trajectory planning. Experiments with Vision‑Language Model agents show a causal disconnect: removing navigation priors significantly harms planning, while removing the chain-of-thought has little effect. The authors propose the Reasoning‑Planning Decoupling Hypothesis and present a training‑free probe to assess an agent’s reliance on priors.

By Xurui Song, Shuo Huai, JingJing Jiang, Jiayi Kong, Jun Luo
arXiv Computer Vision
Oct 12

Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress

The paper introduces VLN-AIEP, a Vision‑Language Navigation framework that integrates instruction‑execution progress awareness into navigation decisions. By linking the visited trajectory to the instruction, it suppresses embeddings of already executed content and matches candidate locations with pending instruction segments. The method employs a landmark‑anchored trajectory representation to encode semantic labels and geometry, enabling progress estimation without explicit supervision and achieving strong results on R2R, RxR, CVDN, and R2R‑CE benchmarks.

By Jiagan Huang, Zikun Zhou, Hongpeng Wang, Guangming Lu, Jun Yu, Wenjie Pei
arXiv AI
Jun 30

CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigation

arXiv:2606. 28397v1 Announce Type: cross Abstract: Vision-language navigation (VLN) has recently advanced with large language and multimodal models, enabling agents to follow natural-language instructions in unseen environments without training a task-specific navigation policy.

By Shaoxuan Li, Xiangyu Dong, Xiaoguang Ma, Junfeng Chen, Haoran Zhao, Yaoming Zhou
arXiv Computer Vision
Oct 12

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

GuidedNav is a new framework for vision‑and‑language navigation that explicitly shapes the representations linking instructions to actions. It uses subtask and landmark guidance to focus pre‑action attention on the relevant instruction segment and visual regions, and spatial guidance to align internal representations with geometric features of the environment. The method achieves state‑of‑the‑art success rates on R2R‑CE and RxR‑CE val‑unseen, improving by 8.1% and 13.1% respectively, while also providing faster inference and introducing two new annotation‑enriched datasets, Guided‑R2R and Guided‑RxR.

By Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong