arXiv AI
Oct 9

Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

The paper introduces SVC, a parameter‑efficient continual‑learning method for large language models that uses singular‑vector channels as the unit for balancing plasticity and stability. By estimating each channel’s adaptation benefit and forgetting cost from domain‑specific and general‑domain data, SVC selectively updates only the most beneficial channels, employing knee‑based cost screening, Pareto‑front filtering, and Otsu thresholding. Experiments on four LLM families and eight downstream tasks show that SVC preserves pretrained capabilities while achieving strong performance compared to existing PEFT baselines.

By Lingxiang Wang, Hainan Zhang, Liang Pang, Hongwei Zheng, Zhiming Zheng