arXiv:2609. 32499v2 Announce Type: replace-cross Abstract: Continually adapting large language models requires acquiring new knowledge while preserving previously learned capabilities.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2609.26257v1 Announce Type: new Abstract: Continual learning (CL) aims to incrementally acquire knowledge from sequential data while avoiding catastrophic forgetting. Recently, prompt tuning ha...
arXiv:2507. 14725v4 Announce Type: replace-cross Abstract: Prompt-based continual learning (CL) offers a parameter-efficient way to adapt large language models (LLMs) across task sequences.
arXiv:2605.12484v3 Announce Type: replace Abstract: Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating model parameters force...
The paper investigates how different prompt adaptation techniques—discrete prompt optimization, prompt tuning, and prefix tuning—affect the internal processing of dense and mixture-of-experts language models. It finds that the adaptation location determines how task information reaches the output: text prompts mainly influence response construction, input-level prompts modify content representations, and layer-wise prefixes exert direct control via attention. In MoE models, these differences also lead to distinct expert‑routing patterns, while generic representation‑level metrics fail to reliably distinguish effective adaptations from placebo effects.
The paper introduces Evolving Functional REpertoires (EFRE), a method that replaces a single prompt with a dynamic set of functions to enable large language models to learn new tasks sequentially without forgetting. EFRE refines existing functions when updates are compatible and creates new ones when updates conflict, thereby mitigating catastrophic forgetting and overfitting. Experiments on a three‑task continual‑learning stream show EFRE outperforms GRPO by 7.50 percentage points and reduces performance drop on a subsequent task by only 1.56 percentage points compared to base prompt optimization.
The paper introduces Skill Schema Transport (SST), a post‑training framework that mitigates catastrophic forgetting in continual domain adaptation by translating new domain knowledge into a model’s existing skill space. SST abstracts each example type into a reusable skill schema, preserving essential operations while removing domain‑specific surface forms, thereby turning knowledge updates into procedural reuse rather than trajectory memorization. Experiments on two 3B instruction‑tuned backbones show that SST consistently reduces forgetting and improves generalization, achieving up to a 5.75‑point performance gain and a dramatic drop in maximum forgetting from 42.41 to 7.10 points on the Qwen‑3B MATH stream.