arXiv:2609.22566v1 Announce Type: cross
Abstract: Knowledge distillation (KD) aims to compress high-performance teacher LLMs into lightweight students. However, distilled students often exhibit subst...
By Dileesha Kannangara, Sanghamitra Dutta
arXiv:2606. 25927v1 Announce Type: cross Abstract: As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding.
By Luyang Fang, Haoran Lu, Yongkai Chen, Wenxuan Zhong, Ping Ma
arXiv:2609.13199v1 Announce Type: new
Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...
By Dawen Jiang, Zhishu Shen, Zeyu Liu, Tiehua Zhang
Interpretable AI with Local Distillation proposes a method where a black‑box teacher model guides a regularized linear student model at each query point. The teacher defines locality by upweighting training observations with similar predicted outcomes and anchors the fit with its own prediction at the query point, treated as a pseudo‑observation. By adding Gaussian randomization and refitting, the approach identifies reliable features and stable subgroups, achieving near‑teacher accuracy while producing sparse, locally interpretable linear models.
By Erin Craig, Yiling Huang, Snigdha Panigrahi
arXiv:2509. 09371v2 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions.
By Zitao Wang, Nian Si, Molei Liu
arXiv:2606. 12171v1 Announce Type: cross Abstract: Knowledge Distillation (KD) and mixup have proven effective at inducing smoothness in class boundaries; KD captures inherent class relationships in probability distributions, and mixup enforces them through convex combinations of inputs.
By Jos\'e Medina, Paul Honeine, Abdelaziz Bensrhair, Amnir Hadachi