arXiv AI

Parameter-Efficient Continual Fine-Tuning: A Survey

arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.

arXiv AI

Aug 11

Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages

arXiv:2603. 12658v2 Announce Type: replace-cross Abstract: Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs.

By Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin

llms fine-tuning benchmarks safety

arXiv AI

6d ago

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

arXiv AI

Jul 8

To Retain or to Adapt? Generalizing Continual Learning

arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.

By Giulia Lanzillotta, Mandana Samiei, Doina Precup, Razvan Pascanu, Claire Vernade

computer-vision reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 14

Attribution-Guided Continual Learning for Large Language Models

arXiv:2605. 05285v2 Announce Type: replace Abstract: Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks.

By Yazheng Liu, Yuxuan Wan, Rui Xu, Xi Zhang, Sihong Xie, Hui Xiong

llms fine-tuning

arXiv AI

Aug 7

Continual Learning in Transition

arXiv:2608. 06216v1 Announce Type: cross Abstract: Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.

By Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua

arXiv Machine Learning

Jul 7

The Bayesian Approach to Continual Learning: An Overview

arXiv:2507. 08922v3 Announce Type: replace-cross Abstract: Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps.

By Tameem Adel

fine-tuning benchmarks

arXiv Machine Learning

Jul 28

Gradient-Free Continual Learning

arXiv:2504. 01219v2 Announce Type: replace Abstract: Neural networks are notorious for forgetting old skills when taught new ones - a problem known as catastrophic forgetting.

By Grzegorz Rype\'s\'c

benchmarks safety

arXiv Machine Learning

Jul 10

When Does Continual Learning Require Learning

arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?

By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jun 17

Position: Modular Memory is the Key to Continual Learning Agents

arXiv:2603. 01761v2 Announce Type: replace-cross Abstract: Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute.

By Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov, Lucas Caccia, Antonio Carta, Laurent Charlin, Barbara Hammer, Tyler L. Hayes, Timm Hess, Christopher Kanan, Dhireesha Kudithipudi, Xialei Liu, Vincenzo Lomonaco, Jorge Mendez-Mendez, Darshan Patil, Ameya Prabhu, Elisa Ricci, Tinne Tuytelaars, Gido M. van de Ven, Liyuan Wang, Joost van de Weijer, Jonghyun Choi, Martin Mundt, Rahaf Aljundi

llms agents

arXiv AI

Jun 24

Fast and Slow Variational Continual Learning

arXiv:2606. 24007v1 Announce Type: cross Abstract: Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation.

By Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan, Siddharth Swaroop, Thomas M\"ollenhoff, Martin Mundt

llms fine-tuning

arXiv AI

Jun 6

Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments

arXiv:2606. 05661v1 Announce Type: new Abstract: Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it.

By Parth Asawa, Christopher M. Glaze, Gabriel Orlanski, Ramya Ramakrishnan, Benji Xu, Asim Biswal, Vincent Sunn Chen, Frederic Sala, Matei Zaharia, Joseph E. Gonzalez

llms agents benchmarks

arXiv AI

Aug 7

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605. 20247v2 Announce Type: replace-cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).

By Yang Liu, Toan Nguyen, Flora D. Salim

llms fine-tuning multimodal benchmarks safety

Related stories