← Back to all news
arXiv AI By Qixin Sun, WenLong Gao, Xiansheng Chen, Zhongyuan Wang, Si Liu, Pengwei Wang

LTM: Latent Transition Based In-Context Memory for Vision-Language-Action Models

Read the original on arXiv AI →

The paper introduces Latent Transition Memory (LTM), a method that stores interaction history as compact latent transitions between consecutive observations for vision‑language‑action models. Implemented as LTM‑π on the π₀.₅ architecture, it uses a frozen latent action encoder, a lightweight adapter, and causal KV caching to enable incremental inference without external reasoning models or memory‑specific annotations. Experiments on RoboMME and LIBERO‑Mem show that LTM‑π achieves state‑of‑the‑art success rates (58.54% and 80.5% respectively), and scaling the dataset or applying a 200‑token memory budget still yields high performance, demonstrating the method’s scalability and efficiency in real‑world robotic tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 9

Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models

The paper introduces Reasoning and Flow Memory (FLOWMEM), a vision‑language‑action model that reuses successful latent reasoning steps instead of discarding them after each policy query. FLOWMEM dynamically retrieves and recomposes compatible latent fragments to form a reasoning route that aligns with the temporal structure of prior successful computations, then refines this route with current visual and proprioceptive data before generating actions. Experiments on RoboMME and LIBERO‑Plus show that FLOWMEM improves success rates by 1.7 and 4.1 percentage points over memory‑free policies, demonstrating the benefit of reusing latent computation for closed‑loop control.

By Hongyu Shi, Sen Zhao, Zuyu Zhang, Lifeng Shen, Ding Zou, Xinyu He, Xu Zhang, Qinghua Zhang
roboticsmultimodal
More like this →
arXiv AI
Jul 24

VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory

arXiv:2603. 04910v2 Announce Type: replace-cross Abstract: Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory.

By Yuheng Lei, Zhixuan Liang, Hongyuan Zhang, Ping Luo
llmsragdiffusionroboticsmultimodalbenchmarks
More like this →
arXiv AI
Sep 24

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

MemBodied introduces a fixed‑size episodic memory for Vision‑Language‑Action models, comprising an associative state that tracks interactions across policy calls and an episode anchor that stores a compact representation of the initial scene. By conditioning action generation on these memory components instead of raw past observations, MemBodied reduces context bloat and inference latency. In five memory‑dependent RMBench tasks, it outperforms stateless and vanilla recurrent policies by significant margins, and achieves a 90.6% success rate on the LIBERO‑Long suite, improving over the baseline by 5.4%.

By Tej Deep Pala, Navonil Majumder, Bryce Goh, Raphael Yee, Jianfei Yang, Liming Chen, Soujanya Poria
roboticsefficiencymultimodal
More like this →