MemoryVLN introduces a memory-driven navigation system that compresses long navigation histories and reuses past spatial experience. It uses forgetting-curve-inspired temporal sampling for long sequence compression, an ego-aware memory representation that fuses 2D semantic, 3D geometric, and 6D pose tokens, and a hierarchical memory bank for in-episode and context-level memory management. The system retrieves top‑K relevant memories to augment current visual tokens, achieving significant improvements on R2R‑CE and RxR‑CE benchmarks.
By Xueyao Wei, Yifei Wang, Delun Li, Shaoan Wang, Junzhi Yu, Wenjie Song, Xingyu Chen
Large vision-language models (LVLMs) have shown strong potential for embodied vision-language navigation (VLN), yet their performance degrades on long-horizon trajectories due to context-length limits...
LookStep is a new end‑to‑end framework for Vision‑Language Navigation that integrates Language‑Centric Future State Modeling with an Event‑Driven Rolling Memory. It uses language labels to predict coarse navigation progress and future states for candidate actions, and autonomously decides which observations to store in a bounded memory with semantic roles. Empirical results show that LookStep outperforms existing methods on VLN‑CE tasks, achieving a 49.7% success rate on R2R‑CE Val‑Unseen while improving memory efficiency and reducing data requirements.
By Kun-Yang Yu, Yingzhe Li, Hongyu Xu, Shi-Yu Tian, Zhi Zhou, Yang Chen, Ming Yang, Sheng Wang, Qing Yu, Lan-Zhe Guo, Yu-Feng Li
arXiv:2610.35066v1 Announce Type: new
Abstract: Vision-language navigation (VLN) requires an embodied agent to follow language instructions through 3D environments. Maintaining persistent episodic me...
By Yan Wang, Jiangyong Huang, Baoxiong Jia, Siyuan Huang
arXiv:2604. 17473v3 Announce Type: replace-cross Abstract: Vision-Language Navigation(VLN) requires an agent to navigate through 3D environments by following natural language instructions.
By Kangyi Wu, Pengna Li, Kailin Lyu, Xi Lin, Lin Zhao, Qingrong He, Jinjun Wang, Jianyi Liu
arXiv:2610.27104v1 Announce Type: new
Abstract: Reliable language-guided perception is essential for embodied agents, where questions about objects, events, and their locations must be answered from...
By Victoria Khoruzhaya, Maksim Kalmykov, Tuong Do Minh, Tatiana Zemskova, Yixin Zhu, Dmitry Yudin
SpatialMem is a memory-based system that fuses 3D geometry, semantics, and language into a single queryable representation. Starting from casually recorded egocentric RGB video, it reconstructs metrically scaled interior scenes, identifies structural 3D anchors (walls, doors, windows), and links open‑vocabulary object nodes to evidence patches, visual embeddings, and two‑layer descriptions at 3D coordinates, enabling spatial‑relation reasoning, language‑directed retrieval, and QA without special sensors. The system is evaluated at scale on public benchmarks, achieving competitive scores on VSI‑Bench, OpenEQA, and ScanNet, while maintaining a compact index of about 14 KB per scene.
By Xinyi Zheng, Yunze Liu, Chi-Hao Wu, Fan Zhang, Hao Zheng, Wenqi Zhou, Walterio W. Mayol-Cuevas, Junxiao Shen
arXiv:2609.09396v1 Announce Type: new
Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...
By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali
arXiv:2510. 01483v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query.
By Mohamad Al Mdfaa, Svetlana Lukina, Timur Akhtyamov, Arthur Nigmatzyanov, Dmitrii Nalberskii, Sergey Zagoruyko, Gonzalo Ferrer
TimeBlind is a diagnostic benchmark designed to evaluate fine‑grained spatio‑temporal compositionality in video large language models (LLMs). It categorizes temporal understanding into three levels—atomic event recognition, event property characterization, and reasoning about event interdependencies—and uses a minimal‑pairs paradigm where video pairs share identical static content but differ only in temporal structure. Across 20 state‑of‑the‑art MLLMs tested on 600 curated instances, the best model achieved only 48.2% instance accuracy, far below human performance of 98.2%, highlighting a reliance on static visual shortcuts rather than true temporal reasoning.
By Baiqi Li, Kangyi Zhao, Ce Zhang, Chancharik Mitra, Jean de Dieu Nyandwi, Gedas Bertasius
The paper introduces 4DMemory, a benchmark for long‑horizon spatiotemporal memory that includes the LongVideo dataset—about 900 hours of video with over 5,000 manually reviewed questions—and the 4DMemoryBench. It also presents the 4DMemory Harness, a system that combines four complementary skills, more than 22 specialized tool models, and a retrieval‑augmented generation component. Evaluation of 13 leading models shows that standalone foundation models perform poorly on long‑horizon tasks, while the 4DMemory Harness boosts performance by over 50% and reveals a performance drop of more than 10% compared to previous benchmarks.
By Yanhao Jia, Yi Zhu, Yiran Zhong, Weigao Sun, Feida Zhu, Pengkun Jiao, Sihan Yang, Ming Ma, Steven Hoi, Erik Cambria
arXiv:2607. 25487v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models translate natural-language commands into robot action sequences, but leading systems on the LIBERO-Plus robustness benchmark use three- to seven-billion-parameter backbones whose memory demands can exceed embedded robotic budgets.
By Minhyeok Lee, Chiyoung Kim, Chanhoe Gu, Seongrok Kim, Sanghyuk Roy Choi, Donghwan Hwang, Donghun Ryu, Seokhyun Kim