arXiv Computer Vision
Task Relevance-Driven Prompt Expansion for Federated Continual Learning
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
Sep 23
Information-Theoretic Decoupled Prompt Tuning for Continual Learning
arXiv:2609.26257v1 Announce Type: new Abstract: Continual learning (CL) aims to incrementally acquire knowledge from sequential data while avoiding catastrophic forgetting. Recently, prompt tuning ha...
arXiv AI
Sep 28
Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models
The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.
arXiv Machine Learning
6d ago
FedSEAM: Federated Continual Learning of LLMs via Subspace Expert Agreement Merging
FedSEAM is a federated continual learning framework that trains large language models across multiple clients without sharing data, using sparse mixture-of-experts to adapt selected parameters on a frozen backbone. It introduces task-aware tile selection for new tasks and a subspace-based compatibility test to decide which expert tiles to reuse or replace. On the server, Subspace Expert Agreement Merging (SEAM) aligns client updates only along agreed subspaces, while additional protections mitigate routing drift and merge erasure, preserving prior knowledge during training.
arXiv Machine Learning
Jul 14
Attribution-Guided Continual Learning for Large Language Models
arXiv:2605. 05285v2 Announce Type: replace Abstract: Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks.
arXiv Machine Learning
Jul 17
Knowledge-Aware Evolution for Task-Free Streaming Federated Continual Learning with Arbitrary Class Overlap
arXiv:2601. 19788v2 Announce Type: replace Abstract: Federated Continual Learning (FCL) leverages inter-client collaboration to better balance new knowledge acquisition and old knowledge retention on non-stationary data.
arXiv Machine Learning
Aug 28
Unifying Detection and Adaptation in Task-Free Continual Learning
The paper introduces FiUni, a Fisher-guided unified framework that performs batch-level task detection and parameter-efficient continual adaptation for large language models. By exploiting orthogonality in the Fisher information matrix’s Kronecker-Factored Approximate Curvature subspaces, FiUni constructs frozen subspaces to guide low-rank adaptation and matches incoming batch windows to historical subspaces. This approach allows the model to decide whether to reuse, expand, or create new subspaces, balancing knowledge sharing and task isolation while achieving competitive performance with fewer trainable parameters.