arXiv AI August 11, 2026 By Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin

Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages

arXiv:2603. 12658v2 Announce Type: replace-cross Abstract: Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 14

Attribution-Guided Continual Learning for Large Language Models

arXiv:2605. 05285v2 Announce Type: replace Abstract: Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks.

By Yazheng Liu, Yuxuan Wan, Rui Xu, Xi Zhang, Sihong Xie, Hui Xiong

llms fine-tuning

arXiv AI

Jul 22

Parameter-Efficient Continual Fine-Tuning: A Survey

arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.

By Eric Nuertey Coleman, Luigi Quarantiello, Ziyue Liu, Qinwen Yang, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco

fine-tuning benchmarks

arXiv AI

Aug 7

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605. 20247v2 Announce Type: replace-cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).

By Yang Liu, Toan Nguyen, Flora D. Salim

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 28

MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

arXiv:2607. 22556v1 Announce Type: new Abstract: Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments.

By Dong Li, Yanchi Liu, Xujiang Zhao, Wei Cheng, Zhengzhang Chen, Xintao Wu, Zhong Chen, Chen Zhao, Haifeng Chen

llms

arXiv Machine Learning

Jul 10

When Does Continual Learning Require Learning

arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?

By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

6d ago

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

Related stories

Attribution-Guided Continual Learning for Large Language Models

Parameter-Efficient Continual Fine-Tuning: A Survey

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

When Does Continual Learning Require Learning

Representation Finetuning for Continual Learning