MemoryVLN introduces a memory-driven navigation system that compresses long navigation histories and reuses past spatial experience. It uses forgetting-curve-inspired temporal sampling for long sequence compression, an ego-aware memory representation that fuses 2D semantic, 3D geometric, and 6D pose tokens, and a hierarchical memory bank for in-episode and context-level memory management. The system retrieves top‑K relevant memories to augment current visual tokens, achieving significant improvements on R2R‑CE and RxR‑CE benchmarks.
Large vision-language models (LVLMs) have shown strong potential for embodied vision-language navigation (VLN), yet their performance degrades on long-horizon trajectories due to context-length limits...
arXiv:2604. 17473v3 Announce Type: replace-cross Abstract: Vision-Language Navigation(VLN) requires an agent to navigate through 3D environments by following natural language instructions.
The paper introduces Linguistic Trajectory Encoding (LTE), a hybrid representation that compresses dynamic object motion histories using natural language descriptions, sparse spatial anchors, and visual anchors. LTE adapts compression to motion complexity by anchoring periods without reliable observations to the last seen location while preserving accuracy with geometric waypoints and linguistic descriptions. Evaluated on the newly constructed Spatial Memory Benchmark (SMB) from EgoLife multi‑day recordings, LTE achieves 45.3 % success in semantic trajectory retrieval and 48.7 % in long‑horizon object retrieval, outperforming prior structured‑memory and VLM baselines, and compresses trajectories 8.7×–26.1× with sub‑second query latency on 24‑hour video.
arXiv:2609.28236v1 Announce Type: new Abstract: Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounte...
LookStep is a new end‑to‑end framework for Vision‑Language Navigation that integrates Language‑Centric Future State Modeling with an Event‑Driven Rolling Memory. It uses language labels to predict coarse navigation progress and future states for candidate actions, and autonomously decides which observations to store in a bounded memory with semantic roles. Empirical results show that LookStep outperforms existing methods on VLN‑CE tasks, achieving a 49.7% success rate on R2R‑CE Val‑Unseen while improving memory efficiency and reducing data requirements.
UniMem is a framework that unifies visual and textual memory with low‑level control in a single Vision‑Language‑Action model, using an event classifier, keyframe encoder, and caching to manage memory efficiently. It addresses non‑Markovian tasks that require remembering past scenes, avoiding the memory bottleneck of separate VLMs and the performance drop from fixed‑interval frame sampling. Across five simulation and four hardware tasks, UniMem outperforms fixed‑interval sampling (93.4% vs. 68.2%) and hierarchical VLM‑VLA baselines (80.0% vs. 43.5%) while offering faster inference and a simpler training pipeline.
LT-Mem introduces a volatility‑aware memory evolution framework for lifelong scene understanding, combining spatially aligned instance‑level 3D perception with temporal reasoning. It uses a multi‑session SLAM backbone, a reasoning layer that scores evidence and selects memory actions, and a Tri‑Memory structure (Live, Delta, Meta) to preserve current states and event histories. The accompanying LT‑VQA dataset provides multi‑session recordings, persistent identity annotations, and temporal QA pairs, and experiments show LT‑Mem outperforms baselines while using far fewer tokens.
Building memory is essential for long-horizon planning in zero-shot embodied navigation. Detector-centric scene graphs often compress observations into sparse nodes, discarding fine-grained visual evidence and accumulating noise, while 3D reconstruction-based methods remain computationally prohibitive.
The paper introduces RePlanMem-VLA, a selective visual memory system for Vision‑Language‑Action models that uses a policy’s own plan revisions as an online memory‑write signal. By detecting unusually large disagreements between temporally aligned action predictions, the method identifies task‑relevant events and updates a bounded visual memory while suppressing redundant frames. Experiments on MIKASA‑Robo‑VLA tasks and real‑robot replay data show that RePlanMem‑VLA retains essential observations in every episode and achieves high ROC‑AUC scores for event detection.
arXiv:2610.01192v1 Announce Type: new Abstract: Streaming vision-language models must process continuously growing video streams under a bounded compute budget, creating a persistent tension between...
arXiv:2609.23601v1 Announce Type: new Abstract: Long-video understanding must capture transient visual evidence under strict token budgets, yet existing methods compress frames, append memory tokens,...
arXiv:2610.14406v1 Announce Type: cross Abstract: It is widely appreciated that robotic policies require long-term memory to perform complex tasks. However, directly conditioning policies on all past...