Changepoint-Aware World Models (CAWM) is a DreamerV3 agent that detects abrupt dynamics shifts in a robot’s environment using an online CUSUM test on internal prediction error. Upon detection, CAWM selectively forgets stale replay data while preserving the learned representation, enabling rapid recovery from shifts such as doubled gravity or halved actuator gain. Experiments on simulated locomotion show CAWM recovers faster than passive retraining and outperforms a baseline that respawns a fresh dynamics model, achieving significant return gains in the first 30k post‑shift frames.
By Everest Yang
The paper introduces Space-sampled Value Decay (SsVD), a forgetting mechanism designed for non-stationary reinforcement learning where the environment can drift at every timestep. SsVD selectively pulls value estimates of randomly chosen state-space elements toward a baseline, discarding outdated information without requiring reset or change-point detection. Integrated into Soft Actor Critic and Deep Q-Networks, SsVD outperforms its base algorithms across six non-stationary environments and can also promote optimism in hard-exploration tasks.
By Felix St\"orck, Philipp Hartmann, Fabian Hinder, Klaus Neumann, Barbara Hammer
arXiv:2608. 13600v1 Announce Type: cross Abstract: We study Evolution Strategies (ES) for continual control, where agents must adapt to changing tasks without forgetting previous ones.
By Nicola Pitzalis, Eleni Nisioti, Antonio Carta, Davide Bacciu, Andrea Cossu
The paper introduces a compositional continual learning benchmark for world models in robot manipulation, designed to isolate knowledge reuse from learning speed and capacity. Tasks are curated to combine previously seen action and perception components, allowing analysis of how different modalities affect reuse. Experiments show that modular world models better balance reuse and forgetting than conventional methods, yet none fully solve the challenge, highlighting the need for models explicitly built to reuse knowledge without forgetting.
By Haoyu Zhou, Joe Watson, Anson Lei, Ingmar Posner
arXiv:2607. 01111v1 Announce Type: cross Abstract: Robot policies inevitably encounter failures when deployed in real environments.
By Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski, Jeff Schneider
arXiv:2603. 11395v3 Announce Type: replace-cross Abstract: Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks.
By Abdulaziz Alyahya, Abdallah Al Siyabi, Markus R. Ernst, Luke Yang, Levin Kuhlmann, Gideon Kowadlo
The paper introduces Uncertainty-Driven Replay Memory (UDRM), a new experience replay buffer for reinforcement learning that prioritizes storing transitions with high uncertainty estimates. Unlike traditional buffers that rely on temporal difference error or transition distributions, UDRM updates its contents based on uncertainty derived from the RL model during training. Experiments show that this uncertainty-aware buffer leads to higher rewards during training compared to other uncertainty-aware RL frameworks.
By Sheeraja Rajakrishnan, Alexander G. Ororbia, Travis Desell, Daniel E. Krutz
arXiv:2607. 04364v1 Announce Type: new Abstract: Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks.
By Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou, Bo Ye, Jian Zhao, Min-Ling Zhang, Tong Wei
arXiv:2609.17141v1 Announce Type: cross
Abstract: Traversability prediction is a critical component of autonomous navigation in unstructured environments, where complex and uncertain robot-terrain in...
By Hojin Lee, Yunho Lee, Daniel A Duecker, Cheolhyeon Kwon
arXiv:2505. 18347v3 Announce Type: replace-cross Abstract: Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation.
By Mohamed A. Mohamed, Kateryna Nekhomiazh, Vedant Vyas, Marcos M. Jose, Andrew Patterson, Marlos C. Machado
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai
Headroom-Drift Replay is a replay control primitive designed for GRPO that separates reuse into two decisions: Headroom ranks stored groups by remaining learning value, while Drift gates them by compatibility with the current policy. The method keeps the fresh on‑policy stream unchanged and adds no auxiliary generation or training machinery. Across mathematical reasoning, multimodal reasoning, and Agentic Search benchmarks, it outperforms naive replay and matches or exceeds broader replay methods on Avg Mean@32, delivering comparable quality at materially lower wall‑clock time in Agentic Search.
By Hyun Bin Park, Du-Seong Chang