The paper proposes ten desiderata for designing and evaluating reinforcement‑learning agents that learn continuously over a lifetime. It reviews existing approaches against these criteria, noting that no single agent has yet integrated all ten. Additionally, it introduces a compact persistent‑world scenario and a benchmark to test multiple behavioral capabilities within one continuous lifetime.
arXiv:2506. 14990v3 Announce Type: replace Abstract: Benchmarks play a central role in reinforcement learning (RL) research, yet their computational constraints often shape what is studied.
arXiv:2505. 18347v3 Announce Type: replace-cross Abstract: Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation.
Memory-Centric Continual Learning for AI Agents proposes a framework that redefines agent memory as a driver of future actions rather than a passive retrieval cache. It introduces a three-axis taxonomy—substrates, assets, and policies—to structure memory, a two-loop lifecycle for updating memory state and refining management policies, and an evaluation protocol that isolates genuine behavioral gains from simple context expansion. The paper also presents a memory scaling framework to model how performance scales with experience, storage, and compute across non-parametric and parametric memory.
The paper introduces Evolving Functional REpertoires (EFRE), a method that replaces a single prompt with a dynamic set of functions to enable large language models to learn new tasks sequentially without forgetting. EFRE refines existing functions when updates are compatible and creates new ones when updates conflict, thereby mitigating catastrophic forgetting and overfitting. Experiments on a three‑task continual‑learning stream show EFRE outperforms GRPO by 7.50 percentage points and reduces performance drop on a subsequent task by only 1.56 percentage points compared to base prompt optimization.
arXiv:2606. 04029v1 Announce Type: cross Abstract: Reinforcement Learning (RL) has received increasing attention and adoption in real-world use cases.
arXiv:2604. 15414v2 Announce Type: replace-cross Abstract: Continual reinforcement learning must balance retention with adaptation, yet many methods still rely on \emph{single-model preservation}, committing to one evolving policy as the main reusable solution across tasks.
EmbodiedCLBench is the first benchmark designed to evaluate continual learning in embodied simulation environments, focusing on self‑evolving agents. The benchmark uses compositional generalization to test whether agents can learn basic tasks and then recombine those skills to solve more complex, unseen tasks. Experiments across various agent harnesses and learning methods reveal that while basic task experience improves performance on advanced tasks, this benefit does not scale with more learning samples and current agents fail to extract additional useful information.
The paper introduces HaMo, a Harness-Model collaborative framework designed for long-horizon continual learning. HaMo combines fast non‑parametric experience accumulation—distilling trajectories into strategy‑level experience cards for immediate retrieval—with slow parametric adaptation, training the model to interpret, ground, and execute these strategies across changing contexts. A retrieval‑coverage‑based selection mechanism identifies compact, complementary trajectories for parameter updates, and experiments demonstrate that HaMo outperforms both harness‑only and model‑only approaches in within‑domain and cross‑domain settings, improving retention and adaptation.
arXiv:2611.07327v1 Announce Type: cross Abstract: Continual Reinforcement Learning (CRL) agents struggle to adapt to new situations while retaining prior knowledge, leading to a stability-plasticity...
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
The paper introduces Harness Continual Learning (HCL), a paradigm where an agent’s state evolves through prompts, memories, tools, skills, and routing rules while keeping the underlying foundation model frozen. HCL defines harness-level forgetting and proposes a guarded evolution process involving a Continual Optimizer and Evaluator to ensure improvements without losing prior behavior. Experiments across textual reasoning, multimodal perception, and open‑world interaction show over 10% performance gains and demonstrate how the stability–plasticity trade‑off can be explicitly tuned.
arXiv:2610.17726v1 Announce Type: cross Abstract: Large language model (LLM) agents excel at complex, multi-step tasks but suffer from catastrophic forgetting during continual learning--acquiring new...