arXiv AI By Nishit Anand, Ramani Duraiswami, Dinesh Manocha

What Should World Models Forget? Stratified Retention for Continual Adaptation

Read the original on arXiv AI →

The paper argues that continual learning for world models must differentiate between knowledge that should never be revised—such as physics and object permanence—and knowledge that should be updated when the environment changes. It critiques existing forgetting metrics and benchmarks for failing to capture this distinction, and proposes a differential retention approach that tracks invariant regression testing and revision latency throughout the adaptation process.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Jul 29

ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

Large language models (LLMs) have demonstrated strong capabilities in knowledge acquisition and reasoning, yet their ability to retain previously acquired knowledge under repeated updates remains insufficiently understood. Existing evaluation paradigms primarily focus on single-step reasoning or static knowledge editing, which fail to capture the temporal dynamics of knowledge retention and degradation during continual model modification.

arXiv AI
Aug 19

When to Review: Spaced Repetition for Continual Pre-Training of Language Models

The paper introduces Spaced Repetition Training (SRT), a continual learning framework that schedules sample rehearsal using the SM-2 algorithm. SRT tracks per-example review states and maps perplexity to recall quality, allowing the training loop to decide which examples to replay and when. Experiments on Wikipedia and code corpora show that SRT improves the stability‑plasticity trade‑off, recovers 5–37 percentage points of lost old‑knowledge accuracy, and preserves benchmark performance better than naive continual pre‑training or uniform replay.

By Alankar Atreya, Devesh Batra, Yoages Kumar Mantri, Geremy Bantug, Greig A Cowan, Raad Khraishi