arXiv AI

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

MemBodied introduces a fixed‑size episodic memory for Vision‑Language‑Action models, comprising an associative state that tracks interactions across policy calls and an episode anchor that stores a compact representation of the initial scene. By conditioning action generation on these memory components instead of raw past observations, MemBodied reduces context bloat and inference latency. In five memory‑dependent RMBench tasks, it outperforms stateless and vanilla recurrent policies by significant margins, and achieves a 90.6% success rate on the LIBERO‑Long suite, improving over the baseline by 5.4%.

arXiv AI
Jul 24

VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory

arXiv:2603. 04910v2 Announce Type: replace-cross Abstract: Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory.

By Yuheng Lei, Zhixuan Liang, Hongyuan Zhang, Ping Luo
arXiv AI
Sep 12

2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

The paper introduces 2AM, a system that separates memory and action execution in long‑horizon robot manipulation. 2AM stores task memory exclusively in a multimodal Agent, while a single RGB‑based, stateless Action Model performs motion based on language and optional 2D hints. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion—over 61 points higher than the best baseline—demonstrating that agent‑side memory and precise steering of the Action Model can substantially improve performance.

By Yutong Hu, Fengjiao Chen, Xuezhi Cao, Renaud Detry
Hugging Face Trending Papers
Sep 10

2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

The paper introduces 2AM, a system that keeps task memory solely within a multimodal Agent while using a single RGB‑based, stateless Action Model to execute motions. By compiling interaction history into subtask language and optional 2D grasp/place/move hints, the Agent steers the Action Model, which is trained to tolerate imperfect guidance through dropout, noise, and jitter. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion without depth, geometry, or planners, vastly outperforming the best baseline.

arXiv AI
Aug 5

Gated Memory Policy: In-Context Memorization and Adaptation

arXiv:2604. 18933v2 Announce Type: replace-cross Abstract: Robotic manipulation tasks exhibit varying memory requirements, ranging from Markovian tasks that require no memory to non-Markovian tasks that demand in-context memorization of historical information within a single trial or in-context adaptation based on the outcomes of multiple past trials.

By Yihuai Gao, Jeff Jinyun Liu, Shuang Li, Shuran Song
arXiv AI
Aug 26

PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control

PonderPounce introduces a two‑stage system that leverages a multimodal large language model (MLLM) as an episode‑context engine for robot control. The System2 component, Ponder, accumulates observations, demonstrations, and prior cognition in its native causal context, producing subgoal text and reasoning. The System1 component, Pounce, uses the current observation, instruction, and proprioception, receiving only the newest cognition token and its age from Ponder; the pair is jointly trained end‑to‑end without a separate memory module, achieving real‑time action playback and outperforming baseline methods on RoboMME and RoboCasa‑DC benchmarks.

By Suhwan Choi, Jaeyoon Jung, Sungkyung Kim, Yunsung Lee, Youngjae Yu
arXiv Machine Learning
Jun 16

AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention

arXiv:2511. 18960v4 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have shown remarkable progress in embodied tasks recently, but most methods process visual observations independently at each timestep.

By Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, Xiaoyuan Yu