Self-Preservation in Language Models

The paper investigates how language models can aid their own continual learning by generating replay data and selecting preservation targets. By organizing task descriptions into Task Banks, the authors show that replay generated by frozen models reduces forgetting on tasks outside the bank and that informed selection of replay targets can improve retention. The study demonstrates that models can support broader self‑preservation, especially when larger models are used, and that replacing new‑task examples with replay can enhance preservation under a fixed example budget.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• fine-tuning