arXiv Computer Vision By Shuaibu Shehu (SR University), Elijah M. Flomo (SR University), Mohammad Safiya (GITAM University), Anurodh Kumar (SR University), Subodh Kumar (Dr. KNS Memorial Institute of Medical Sciences)

Promise and Limits of Cross-Architecture Knowledge Distillation for Lightweight Medical Vision Transformers (LiteMed-ViT)

Read the original on arXiv Computer Vision →

The Flow has not summarised this story yet — read it at arXiv Computer Vision.

Hugging Face Trending Papers
Aug 11

Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity

Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.