arXiv:2605.06814v2 Announce Type: replace
Abstract: Graph neural networks (GNNs) increasingly rely on sophisticated architectures and training procedures to achieve desirable properties such as high...
By Debolina Halder Lina, Arlei Silva
arXiv:2604. 10882v2 Announce Type: replace-cross Abstract: Graph Neural Network pretraining is pivotal for leveraging unlabeled graph data.
By Yang Yan, Yunxuan Li, Qiuyan Wang, Tianjin Huang, Qiudong Yu
arXiv:2410. 00074v2 Announce Type: replace Abstract: A novel Learning-by-Education Node Community framework (LENC) for Collaborative Knowledge Distillation (CKD) is presented, which facilitates continual collective learning through effective knowledge exchanges among diverse deployed Deep Neural Network (DNN) peer nodes.
By Anestis Kaimakamidis, Ioannis Mademlis, Ioannis Pitas
arXiv:2609.13199v1 Announce Type: new
Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...
By Dawen Jiang, Zhishu Shen, Zeyu Liu, Tiehua Zhang
arXiv:2607. 21885v1 Announce Type: new Abstract: Coarsening-based training for graph neural networks (GNNs), i.
By Guoming Li, Jian Yang, Xukun Wang, Zixiao Wang, Shangsong Liang, Yifan Chen
The paper investigates how knowledge distillation (KD) applied at intermediate layers of a neural network can affect overfitting and model performance. While traditional KD focuses on the final output, this study explores block‑wise KD across eleven datasets, finding that on standard datasets the last block suffices, but on fine‑grained, data‑scarce settings intermediate supervision significantly improves accuracy. The authors also analyze optimal supervision granularity using attention maps, Centered Kernel Alignment, and Grad‑CAM, and examine teacher‑student fine‑tuning strategies.
By Irene Trigueros-Lorca, Leonardo Concepci\'on, Christian Wagner, Isaac Triguero, Daniel Molina