arXiv Machine Learning August 14, 2026 By Qifan Zhang, Yunhui Guo, Yu Xiang

Continual Distillation Learning for Rehearsal-Free Class-Incremental Learning via Decoupled Prompting

Read the original on arXiv Machine Learning →

arXiv:2407. 13911v5 Announce Type: replace-cross Abstract: Prompt-based continual learning has shown strong performance in rehearsal-free class-incremental learning by adapting learnable prompts while freezing a pre-trained Vision Transformer (ViT) backbone.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 10

Self-Distillation Enables Continual Learning

arXiv:2601. 19897v2 Announce Type: replace Abstract: Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models.

By Idan Shenfeld, Mehul Damani, Jonas H\"ubotter, Pulkit Agrawal

llms reinforcement-learning fine-tuning efficiency

arXiv Machine Learning

Jul 3

Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training

arXiv:2607. 01763v1 Announce Type: new Abstract: Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities.

By Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang, Geng Liu, Haiyang Guo, Guo-Sen Xie, Gaofeng Meng, Hongbin Liu, Fei Zhu

reinforcement-learning efficiency

arXiv Machine Learning

Jul 10

When Does Continual Learning Require Learning

arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?

By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 2

Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning

arXiv:2606. 01379v1 Announce Type: new Abstract: While prompt-based parameter-efficient continual learning mitigates catastrophic forgetting by isolating task-specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored.

By Anushka Tiwari, Kaiyi Ji

llms benchmarks

Hugging Face Trending Papers

Jul 2

Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training

Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation emerging as a particularly attractive approach.

reinforcement-learning efficiency

arXiv Machine Learning

Jul 1

Collaborative Knowledge Distillation via a Learning-by-Education Node Community

arXiv:2410. 00074v2 Announce Type: replace Abstract: A novel Learning-by-Education Node Community framework (LENC) for Collaborative Knowledge Distillation (CKD) is presented, which facilitates continual collective learning through effective knowledge exchanges among diverse deployed Deep Neural Network (DNN) peer nodes.

By Anestis Kaimakamidis, Ioannis Mademlis, Ioannis Pitas

agents computer-vision efficiency

Related stories

Self-Distillation Enables Continual Learning

Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training

When Does Continual Learning Require Learning

Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning

Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training

Collaborative Knowledge Distillation via a Learning-by-Education Node Community