arXiv:2605. 20247v2 Announce Type: replace-cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).
By Yang Liu, Toan Nguyen, Flora D. Salim
arXiv:2608.31096v1 Announce Type: cross
Abstract: Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while...
By Yunxiang Fu, Meng Lou, Yizhou Yu
arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.
By Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari
CAMFT is a Conflict‑Aware Mergeable Fine‑Tuning method designed to make task adaptation efficient and merge‑aware for large language models. Unlike existing approaches that only resolve parameter conflicts after fine‑tuning, CAMFT shapes mergeability during training by guiding each task to update sparse coordinates with lower cross‑task conflict. Experiments show that CAMFT outperforms standard fine‑tuning baselines in multi‑task merging scenarios.
By Jingang Zhou, Haiyang Guo, Yuan Ma, Han Zhu, Xu-Yao Zhang
The paper introduces PIECE, a Parameter Importance-Driven Continual Learning method that selectively updates only 0.1% of core parameters to preserve general abilities while learning new domain knowledge. PIECE employs two importance estimators—PIECE‑F using Fisher Information and PIECE‑S combining gradient and curvature information—to guide updates. Experiments on three language models and two multimodal models demonstrate that PIECE maintains general capabilities and achieves state‑of‑the‑art continual learning performance without accessing prior training data or adding parameter overhead.
By Lingxiang Wang, Hainan Zhang, Zhiming Zheng
arXiv:2608. 16345v1 Announce Type: new Abstract: Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks.
By Zhiming Xu, Huiyu Yi, Zhen-Hao Xie, Baile Xu, Furao Shen, Jian Zhao, Suorong Yang