arXiv Machine Learning

Benchmarking World Models for Continual Learning on Compositional Tasks

The paper introduces a compositional continual learning benchmark for world models in robot manipulation, designed to isolate knowledge reuse from learning speed and capacity. Tasks are curated to combine previously seen action and perception components, allowing analysis of how different modalities affect reuse. Experiments show that modular world models better balance reuse and forgetting than conventional methods, yet none fully solve the challenge, highlighting the need for models explicitly built to reuse knowledge without forgetting.

arXiv AI
Oct 12

EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents

EmbodiedCLBench is the first benchmark designed to evaluate continual learning in embodied simulation environments, focusing on self‑evolving agents. The benchmark uses compositional generalization to test whether agents can learn basic tasks and then recombine those skills to solve more complex, unseen tasks. Experiments across various agent harnesses and learning methods reveal that while basic task experience improves performance on advanced tasks, this benefit does not scale with more learning samples and current agents fail to extract additional useful information.

By Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, Yiwu Zhong
arXiv Machine Learning
Oct 12

Dream to Remember: Continual Steering of Generalist Robot Policies with World Models

Dream to Remember (D2R) is a continual learning framework that steers a frozen generalist robot policy toward new tasks while preserving previously acquired skills. It uses a pretrained world model to rehearse past tasks through imagined rollouts, eliminating the need for replay buffers or new data. Experiments on CALVIN, LIBERO, and a real Franka robot show that D2R can learn multiple tasks sequentially—often entirely in imagination—without catastrophic forgetting, outperforming traditional fine‑tuning methods.

By Bahey Tharwat, Iman Nematollahi, Marvin Alles, Xingyuan Zhang, Akshay L Chandra, Abhinav Valada
arXiv AI
Aug 19

Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Teach-and-Grow Learning (TGL) is an agent-centered architecture that transforms a few successful demonstrations into reusable Skill Blocks, enabling a robot to compose, execute, and revise behaviors in new scenes without task-specific policy retraining. The system maintains a Skill Library and structured Experience Memory to capture successes, failures, and repairs, allowing persistent reuse and agent-directed adaptation. Evaluation on the LIBERO benchmark shows state-of-the-art performance, and the authors propose a scaling-law hypothesis suggesting that accumulated reusable experience reduces future-task error and teaching demand following a power-law trend.

By Chang Nie, Zhe Liu, Hesheng Wang
arXiv AI
Oct 5

RIFAR: Reliability and Forgetting-Aware Replay for Continual Robot Learning

RIFAR is a new continual learning method for robots that uses reliability screening and drift-aware replay to mitigate forgetting while keeping storage low. It reconstructs past trajectories from short demonstration prefixes and employs a frozen inverse-dynamics model to verify action‑visual consistency. In experiments on LIBERO suites and real‑world tasks, RIFAR outperforms previous generative replay approaches, achieving high performance with only a small fraction of stored steps.

By Zirong Song, Zheng Lu, Haoran Liao, Wanqi Zhong, Yunhe Ni, Lijie Wang, Xiuying Chen
Hugging Face Trending Papers
Oct 13

ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models

ACCRUE is a continual off‑policy reinforcement learning framework designed for robot foundation models. It keeps lightweight actor and critic networks across tasks, reusing behavioral and value knowledge from previous RL stages while preserving prior behavior through actor distillation on a small set of retained episodes. Experiments on contact‑rich assembly tasks in simulation and on real robots show that ACCRUE speeds up learning of new tasks, nearly eliminates forgetting, and cuts real‑robot training time by 31.0% to 68.5%.

arXiv Machine Learning
Nov 2

Before We Adapt: The Zero-Shot Generalization Bottleneck in Continual World Models

The paper investigates how model-based reinforcement learning can support continual learning by reusing prior knowledge. It introduces a contextual Markov decision process that separates behavioral and physical context, and studies zero‑shot reuse in FourRooms tasks using Dyna‑Q and TD‑MPC. The experiments show that zero‑shot reuse degrades under physical context changes and that this degradation is strongly tied to the observation representation, highlighting representation generalization as a key bottleneck.

By Amir Mesbah, Di Wu