arXiv:2604. 15794v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have achieved remarkable success, underpinning diverse AI applications.
By Chi Liu, Xin Chen, Xu Zhou, Fangbo Tu, Srinivasan Manoharan
arXiv:2607. 15919v1 Announce Type: cross Abstract: Data-free knowledge distillation transfers the knowledge encoded in a teacher model to a student model without access to the original training data.
By Mohamed Amine Kina
arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.
By Qianhan Feng, Wenshuo Li, Tong Lin, Xinghao Chen
arXiv:2603. 14830v3 Announce Type: replace Abstract: Dataset distillation, a training-aware data compression technique, has recently attracted increasing attention as an effective tool for mitigating costs of optimization and data storage.
By Yuri Kinoshita, Naoki Nishikawa, Taro Toyoizumi
Large language models (LLMs) achieve strong performance across many tasks, but their high computational cost limits deployment in resource-constrained environments. Knowledge Distillation (KD) offers a practical solution by transferring knowledge from a teacher model of a larger size to a smaller student model.
arXiv:2402. 14035v4 Announce Type: replace-cross Abstract: Knowledge distillation from foundation models to compact domain models is challenging due to substantial gaps in capacity, architecture, and modality.
By Zichang Liu, Qingyun Liu, Yuening Li, Liang Liu, Anshumali Shrivastava, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao