arXiv AI June 8, 2026 By Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari

Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 7

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605. 20247v2 Announce Type: replace-cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).

By Yang Liu, Toan Nguyen, Flora D. Salim

llms fine-tuning multimodal benchmarks safety

arXiv AI

6d ago

Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models

The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.

By Bing Wang, Changchun Li, Xin-Qiang Cai, Lin Yuanbo Wu, Ximing Li, Gang Niu, Masashi Sugiyama

llms fine-tuning

arXiv Machine Learning

Sep 16

CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework

arXiv:2609.17026v1 Announce Type: new Abstract: Continual learning must balance the learning of new knowledge with the retention of previously learned knowledge to incrementally learn tasks from a da...

By Yunxiang Fu, Meng Lou, Zicheng Liao, Yizhou Yu

fine-tuning efficiency benchmarks

arXiv Machine Learning

Aug 18

Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning

arXiv:2608. 16345v1 Announce Type: new Abstract: Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks.

By Zhiming Xu, Huiyu Yi, Zhen-Hao Xie, Baile Xu, Furao Shen, Jian Zhao, Suorong Yang

arXiv AI

Aug 13

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

arXiv Machine Learning

Jul 28

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

arXiv:2607. 23837v1 Announce Type: new Abstract: Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.

By Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli, Pavan Turaga, Hassan Ghasemzadeh

llms rag fine-tuning benchmarks

One email a morning, machine-written

Related stories

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models

CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework

Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning

Representation Finetuning for Continual Learning

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning