← Back to all news
arXiv AI

Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents

The paper introduces the Trajectory-Grounded Instruction Translator (TGIT), a front‑end that keeps an aerial vision‑and‑language navigation agent frozen while translating short, intent‑driven user instructions into executable commands. By prompting a language model with human‑written style examples, TGIT learns from the agent’s trajectory outcomes and improves success rates from 11.33 % to 37.93 % on weak inputs, and from 11.33 % to 32.51 % on real human instructions. The method also boosts performance on held‑out OpenFly data and provides recovery on CityNav and AirVLN benchmarks.

Related stories

arXiv AI
Oct 16

X-Fly: Learning to Reach and Stop in Full-Chunk Aerial Vision-Language Navigation

X-Fly is a staged training framework designed to improve full‑chunk aerial vision‑language navigation. It uses Recovery DAgger to label learner‑visited states with smooth action chunks and a STOP command, and Flow‑GRPO to jointly refine a chunked flow controller and a Bernoulli STOP policy. The approach achieves 74.18% success rate on test‑seen and 51.73% on test‑unseen, outperforming baseline methods.

By Ruibin Ju, Ruifeng Cai, Min Xu, Lin Wang
llmsagentsmultimodal
More like this →
arXiv AI
Jun 30

CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigation

arXiv:2606. 28397v1 Announce Type: cross Abstract: Vision-language navigation (VLN) has recently advanced with large language and multimodal models, enabling agents to follow natural-language instructions in unseen environments without training a task-specific navigation policy.

By Shaoxuan Li, Xiangyu Dong, Xiaoguang Ma, Junfeng Chen, Haoran Zhao, Yaoming Zhou
ragagentsmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 22

AgriVLN: Vision-and-Language Navigation for Agricultural Robots

AgriVLN introduces a vision‑and‑language navigation benchmark (A2A) tailored for agricultural robots, comprising 1,560 episodes across six scene classes with step‑by‑step instructions and continuous actions. Existing VLN methods perform well on short instructions but poorly on long ones; to address this, the authors propose a Subtask‑Level‑Task (STL) module that decomposes complex instructions into simple subtasks. Integrating STL into the NavGPT backbone yields the AgriVLN method, improving success rate from 0.33 to 0.47 while slightly increasing navigation error from 2.76 m to 2.91 m, and the paper includes ablation and case studies on subtask quantity and scene classes.

By Xiaobei Zhao, Xingqi Lyu, Xin Chen, Xiang Li
roboticsbenchmarks
More like this →
arXiv AI
Jun 8

Think Like a Pilot: Fine-Grained Long-Horizon UAV Navigation

arXiv:2606. 06836v1 Announce Type: cross Abstract: Language-guided UAV agents must execute long-horizon semantic instructions while producing smooth, physically feasible continuous flight commands, yet existing Vision-Language Navigation (VLN) benchmarks typically use discrete or coarse actions and existing UAV Vision-Language-Action (VLA) tasks focus on short, atomic maneuvers.

By Xiangyi Zheng, Xiangyu Wang, Qinan Liao, Zimu Tang, Yue Liao, Dongyue Lyu, Guodong Wang, Junjie Liu, Si Liu
llmsagentsdiffusionmultimodalbenchmarks
More like this →
arXiv Machine Learning
Jun 11

Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering

arXiv:2606. 12299v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone.

By Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy
llmsroboticsfine-tuningmultimodal
More like this →