Hugging Face Trending Papers
Aug 18

DistillPath: An Efficient 22M Distilled Pathology Encoder Approaching Large Foundation Model Performance

DistillPath-KS16 is a 22‑million‑parameter ViT‑S/16 pathology encoder distilled from larger teachers ranging from 86 M to 1.1 B parameters. By training only on the teachers’ final class and patch tokens across 6,000 public slides, it avoids costly pretraining heads and large tile corpora, yet surpasses the kaiko baseline on EVA, HEST, and PLISM benchmarks. The strongest variant, DistillPath-KS16‑Virchow2, achieves a mean EVA score of 0.795—just 0.015 points below the top model—while being 29× smaller and 25× faster.