Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

The paper introduces SVC, a parameter‑efficient continual‑learning method for large language models that uses singular‑vector channels as the unit for balancing plasticity and stability. By estimating each channel’s adaptation benefit and forgetting cost from domain‑specific and general‑domain data, SVC selectively updates only the most beneficial channels, employing knee‑based cost screening, Pareto‑front filtering, and Otsu thresholding. Experiments on four LLM families and eight downstream tasks show that SVC preserves pretrained capabilities while achieving strong performance compared to existing PEFT baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• fine-tuning