Hugging Face Trending Papers July 29, 2026

CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

Read the original on Hugging Face Trending Papers →

Knowledge distillation (KD) enables a compact student model to learn from a powerful teacher and has become an effective paradigm for model compression. The emergence of diverse model architectures has extended KD from homogeneous to heterogeneous settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

efficiency

arXiv Machine Learning

Jul 30

CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

arXiv:2607. 27054v1 Announce Type: new Abstract: Knowledge distillation (KD) enables a compact student model to learn from a powerful teacher and has become an effective paradigm for model compression.

By Fengming Yu, Haiwei Pan, Kejia Zhang, Chunling Chen, Jian Guan, Baoying Ma

efficiency

arXiv Machine Learning

Sep 15

Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation

arXiv:2609.13199v1 Announce Type: new Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...

By Dawen Jiang, Zhishu Shen, Zeyu Liu, Tiehua Zhang

efficiency benchmarks safety

arXiv Machine Learning

Sep 22

Do Student LLMs Inherit OOD Robustness? Invariance-Weighted Distillation for Reliable Knowledge Transfer

arXiv:2609.22566v1 Announce Type: cross Abstract: Knowledge distillation (KD) aims to compress high-performance teacher LLMs into lightweight students. However, distilled students often exhibit subst...

By Dileesha Kannangara, Sanghamitra Dutta

llms nlp efficiency benchmarks

arXiv Machine Learning

Jun 11

Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions

arXiv:2606. 12171v1 Announce Type: cross Abstract: Knowledge Distillation (KD) and mixup have proven effective at inducing smoothness in class boundaries; KD captures inherent class relationships in probability distributions, and mixup enforces them through convex combinations of inputs.

By Jos\'e Medina, Paul Honeine, Abdelaziz Bensrhair, Amnir Hadachi

efficiency

arXiv Machine Learning

Jul 14

Student Capacity Moderates Knowledge Distillation Effectiveness: A Systematic Study Across ResNet Teacher-Student Pairs on CIFAR-10

arXiv:2605. 31191v2 Announce Type: replace Abstract: We investigate how teacher-student capacity relationships modulate knowledge distillation (KD) effectiveness in ResNet-based image classification on CIFAR-10.

By Umut Onur Yasar

computer-vision efficiency

arXiv Machine Learning

Aug 11

A Controlled Study of Feature-Based Knowledge Distillation Across Student Designs

arXiv:2608. 08294v1 Announce Type: new Abstract: Knowledge distillation trains a smaller student to match the outputs of a larger teacher.

By Abhinand Balachandran, Praveen Prashant

efficiency

One email a morning, machine-written

Related stories

CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation

Do Student LLMs Inherit OOD Robustness? Invariance-Weighted Distillation for Reliable Knowledge Transfer

Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions

Student Capacity Moderates Knowledge Distillation Effectiveness: A Systematic Study Across ResNet Teacher-Student Pairs on CIFAR-10

A Controlled Study of Feature-Based Knowledge Distillation Across Student Designs