← Back to all news
arXiv Machine Learning By Chenyuan Chen, Ran Ding, Lechen Zhang, Zewen He, Yoshihiko Nakamura

Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation

Read the original on arXiv Machine Learning →

The paper introduces WholeBodyMem, an object-guided perceptual memory for humanoid loco‑manipulation that records key frames of object events before the robot acts on its goal. Using SAM3 to track task objects, the system selects causally relevant frames within an adaptive budget and feeds them to an action expert via cross‑attention, along with tokens encoding appearance, geometry, and time. The authors also present HuMeGo, a benchmark with simulated and real‑world Unitree G1 tasks, showing that WholeBodyMem significantly improves success rates over memoryless and uniform‑memory baselines—raising policy success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot, while also boosting standard SIMPLE tasks from 70.2% to 86.9%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 15

HWAM: Humanoid World Action Model With Joint State--Action Generation

HWAM (Humanoid World Action Model) jointly generates reference actions and the robot’s post‑execution proprioceptive state, bridging the action–execution gap in hierarchical humanoid systems. It integrates three conditional paths—policy denoising, forward dynamics modeling, and inverse dynamics modeling—to connect policy references, realized motion, and visual outcomes. On the LimX OLI humanoid, HWAM outperforms baselines, achieving a 70.6% success rate on Candy Picking versus 43.3% for Fast‑WAM.

By Yan Yang, Jikun Rong, Minzhao Zhu, Zheyi Zhao, Qirui Hu, Zihan Lan, Weixin Mao, Yinhao Li, Zhen Fu, Hua Chen
roboticsmultimodal
More like this →
arXiv AI
Oct 9

Humanoid World Action Model With Joint State--Action Generation

The paper introduces HWAM, a Humanoid World Action Model that jointly generates reference actions and the robot’s post‑execution proprioceptive state, thereby closing the action‑execution gap in hierarchical humanoid systems. HWAM is trained via three conditional paths—policy denoising, forward dynamics modeling, and inverse dynamics modeling—to align policy outputs, executed motion, and visual outcomes. Experiments on the LimX OLI humanoid show HWAM achieving a 70.6% success rate on Candy Picking, outperforming Fast‑WAM’s 43.3%.

By Yan Yang, Jikun Rong, Minzhao Zhu, Zheyi Zhao, Qirui Hu, Zihan Lan, Weixin Mao, Yinhao Li, Zhen Fu, Hua Chen
roboticsmultimodal
More like this →
arXiv AI
Sep 12

2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

The paper introduces 2AM, a system that separates memory and action execution in long‑horizon robot manipulation. 2AM stores task memory exclusively in a multimodal Agent, while a single RGB‑based, stateless Action Model performs motion based on language and optional 2D hints. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion—over 61 points higher than the best baseline—demonstrating that agent‑side memory and precise steering of the Action Model can substantially improve performance.

By Yutong Hu, Fengjiao Chen, Xuezhi Cao, Renaud Detry
agentsroboticsmultimodal
More like this →