The paper introduces 2AM, a system that separates memory and action execution in long‑horizon robot manipulation. 2AM stores task memory exclusively in a multimodal Agent, while a single RGB‑based, stateless Action Model performs motion based on language and optional 2D hints. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion—over 61 points higher than the best baseline—demonstrating that agent‑side memory and precise steering of the Action Model can substantially improve performance.
By Yutong Hu, Fengjiao Chen, Xuezhi Cao, Renaud Detry
arXiv:2605.13119v2 Announce Type: replace-cross
Abstract: Vision-language-action (VLA) models are effective robot action executors, but they remain limited on long-horizon tasks due to the dual burde...
By Zixing Lei, Changxing Liu, Yichen Xiong, Minhao Xiong, Yuanzhuo Ding, Zhipeng Zhang, Weixin Li, Siheng Chen
The paper introduces 2AM, a system that keeps task memory solely within a multimodal Agent while using a single RGB‑based, stateless Action Model to execute motions. By compiling interaction history into subtask language and optional 2D grasp/place/move hints, the Agent steers the Action Model, which is trained to tolerate imperfect guidance through dropout, noise, and jitter. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion without depth, geometry, or planners, vastly outperforming the best baseline.
AgentDreamer is an agent-guided optimization framework that aligns text instructions with simulated 3D motion by using an off-the-shelf agent to generate a structured semantic motion prior. The prior specifies what should move, in which direction, and in what temporal order, and is represented as parametric functions over temporal chunks to produce dense 3D trajectories. These trajectories are then used to optimize a differentiable MPM simulator so that its rollout matches the prior while respecting scene dynamics, and the framework introduces Motion-IG, an information‑theoretic metric for measuring text‑motion alignment.
By Mijeong Kim, Suho Park, Junha Park, Bohyung Han
arXiv:2610.29866v1 Announce Type: new
Abstract: Geometric structure is essential for robotic manipulation and understanding the 3D world. Most VLA models rely on 2D representations of current observa...
By Qinfan Zhang, Yifan Luo, Haoyuan Shi, Guanlin Liu, Han Dong, Haozhe Shan, Yong Dai, Xiangcong Ren, Xiaozhu Ju
arXiv:2606. 12550v1 Announce Type: cross Abstract: Open-world mapless navigation from sparse language instructions requires resolving underspecified goals and inferring which environmental cues are relevant for reaching the goal.
By Arthur Zhang, Carl Qi, Donne Su, Xiangyun Meng, Amy Zhang, Joydeep Biswas
UniWALM is a unified world-action-language model that jointly learns video, action, and language in a single shared representation, using a mixture-of-transformers backbone with diffusion heads that co-denoise across modalities. By generating language as part of the same pass as action, the model closes a language‑as‑memory loop without an external planner, and introduces Flow Matching with Correction to maintain reliable action streams. Experiments show UniWALM outperforms baselines on compositional pick‑and‑place tasks and memory‑dependent benchmarks, and dramatically improves success on a real‑world Find‑the‑Item task when using generated textual memory.
By Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Barreiros, Manikantan Nambi, Jeroen Vanbaar
arXiv:2608.29537v1 Announce Type: cross
Abstract: Frozen vision-language-action (VLA) policies offer broad manipulation skills but execute open-loop action chunks without tracking task progress, so t...
By Hongbo Gao, Zeyu Ni, Xin Wen, Siyu Xu, Ruifeng Li
arXiv:2609.15683v1 Announce Type: new
Abstract: While In-Context Learning (ICL) enables models to adapt from exemplars without parameter updates, multimodal ICL remains largely underexplored, particu...
By Ziqian Fan, Shibo Xu, Junjie Li, Xiangyu Zhao, Shengyuan Ding, Yifan Yang, Zhenjie Yang, Haodong Duan, Yue Zhou, Zhihang Zhong, Xue Yang
Geometric structure is essential for robotic manipulation and understanding the 3D world. Most VLA models rely on 2D representations of current observations, while geometry-aware approaches often requ...
arXiv:2606. 27581v1 Announce Type: cross Abstract: Current humanoid reinforcement-learning policies excel at free-space motions but struggle with contact-rich tasks, as pure kinematic tracking cannot resolve the physical ambiguities of interacting with objects and uneven terrain.
By Sirui Chen, Shibo Zhao, Zhen Wu, Jiaman Li, Guanya Shi, C. Karen Liu
arXiv:2609.37810v1 Announce Type: cross
Abstract: Vision-language-action and world-action models have demonstrated impressive capabilities in robotics, yet generalization to unseen tasks remains chal...
By Sicheng Xie, Yitong Chen, Haidong Cao, Shunlin Lu, Zuxuan Wu, Yu-Gang Jiang