arXiv Machine Learning July 1, 2026 By Anestis Kaimakamidis, Ioannis Mademlis, Ioannis Pitas

Collaborative Knowledge Distillation via a Learning-by-Education Node Community

Read the original on arXiv Machine Learning →

arXiv:2410. 00074v2 Announce Type: replace Abstract: A novel Learning-by-Education Node Community framework (LENC) for Collaborative Knowledge Distillation (CKD) is presented, which facilitates continual collective learning through effective knowledge exchanges among diverse deployed Deep Neural Network (DNN) peer nodes.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

5d ago

Continual Distillation Learning for Rehearsal-Free Class-Incremental Learning via Decoupled Prompting

arXiv:2407. 13911v5 Announce Type: replace-cross Abstract: Prompt-based continual learning has shown strong performance in rehearsal-free class-incremental learning by adapting learnable prompts while freezing a pre-trained Vision Transformer (ViT) backbone.

By Qifan Zhang, Yunhui Guo, Yu Xiang

llms computer-vision efficiency

arXiv Machine Learning

Aug 4

Progressive$^2$: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression

arXiv:2608. 00129v1 Announce Type: new Abstract: Knowledge distillation (KD) is a widely utilized technique for transferring knowledge from a large model (the teacher) to a smaller model (the student).

By Tiancong Cheng, Ying Zhang, Zhiwen Yu, Yifang Yin, Bin Guo

efficiency

arXiv Machine Learning

Jun 8

Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

arXiv:2606. 07474v1 Announce Type: new Abstract: Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data.

By Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang, Narges Armanfard

efficiency benchmarks safety

arXiv Machine Learning

Jun 9

Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

arXiv:2606. 08978v1 Announce Type: new Abstract: Hypergraph knowledge distillation aims to retain the predictive performance of a hypergraph neural network (HNN) teacher while reducing inference costs through a lightweight student model.

By Joohee Cho, David Yoon Suk Kang, Yunyong Ko

efficiency

arXiv AI

Aug 7

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605. 20247v2 Announce Type: replace-cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).

By Yang Liu, Toan Nguyen, Flora D. Salim

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 16

KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation

arXiv:2606. 16256v1 Announce Type: cross Abstract: Continual learning for pre-trained vision-language models requires balancing three competing objectives: retaining pre-trained knowledge, preserving knowledge from a sequence of learned tasks, and maintaining the plasticity to acquire new knowledge.

By Mao-Lin Luo, Yi-Lin Zhang, Zi-Hao Zhou, Yankun Hong, Xialiang Tong, Mingxuan Yuan, Tong Wei, Min-Ling Zhang

llms computer-vision nlp fine-tuning multimodal

Related stories

Continual Distillation Learning for Rehearsal-Free Class-Incremental Learning via Decoupled Prompting

Progressive$^2$: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression

Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation