Hugging Face Trending Papers August 18, 2026

DistillPath: An Efficient 22M Distilled Pathology Encoder Approaching Large Foundation Model Performance

Read the original on Hugging Face Trending Papers →

DistillPath-KS16 is a 22‑million‑parameter ViT‑S/16 pathology encoder distilled from larger teachers ranging from 86 M to 1.1 B parameters. By training only on the teachers’ final class and patch tokens across 6,000 public slides, it avoids costly pretraining heads and large tile corpora, yet surpasses the kaiko baseline on EVA, HEST, and PLISM benchmarks. The strongest variant, DistillPath-KS16‑Virchow2, achieves a mean EVA score of 0.795—just 0.015 points below the top model—while being 29× smaller and 25× faster.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

benchmarks

arXiv AI

Sep 4

TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

TAP-Path is a task‑adaptive compression framework that restructures a pretrained Virchow2 encoder for histopathology. It selectively removes transformer blocks, prunes patch tokens, and adds a lightweight gated task head, reducing parameters by 24.96% and FLOPs by 35.20% while maintaining high accuracy on a 32‑class benchmark. The method achieves competitive test metrics and improved rare‑class performance, with strong external validation on CPTAC samples.

By Mehedi Hasan, Ashfak Yeafi, Md Khairul Islam

llms efficiency benchmarks

arXiv AI

Aug 25

ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology

arXiv:2608.22066v1 Announce Type: cross Abstract: Attention-based multiple instance learning (ABMIL) using pathology foundation model embeddings is effective for slide-level tasks, but exhaustive inf...

By Duncan Stothers, Ren-Chin Wu, William Lotter

rag

arXiv Machine Learning

Jul 7

TESSERA v2: Scaling Pixel-wise Earth Foundation Models

arXiv:2607. 03949v1 Announce Type: cross Abstract: Pixel-wise Earth-observation (EO) foundation models are now achieving state-of-the-art performance via generated spatial embeddings.

By Zhengpeng Feng, Sadiq Jaffer, Ira Shokar, Jovana Knezevic, Mark Elvers, Clement Atzberger, Robin Young, Aneesh Naik, Niall Robinson, Andrew Blake, David Coomes, Anil Madhavapeddy, Srinivasan Keshav

rag benchmarks

arXiv AI

Jul 28

Cheap Probes Predict Expensive Training in 3D-CT Vision--Language Models

arXiv:2607. 22771v1 Announce Type: cross Abstract: Picking the frozen image encoder for a 3D~CT vision--language model (VLM), together with the token-compression scheme on top of it, is a search over many candidates.

By Renjie Liang

llms rag fine-tuning multimodal benchmarks

arXiv AI

Aug 5

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

arXiv:2608. 03796v1 Announce Type: cross Abstract: Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model is usually recovered through knowledge distillation (KD).

By Bakbergen Ryskulov, Iker Garc\'ia-Ferrero, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Rom\'an Or\'us

llms efficiency benchmarks

arXiv Machine Learning

Aug 4

Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget

arXiv:2608. 00916v1 Announce Type: cross Abstract: Modern reranking recipes---billion-scale cross-encoders, mixture-of-experts (MoE) backbones, and distillation against strong teachers---have outpaced the training infrastructure available to most academic groups.

By Zhichao Xu, Xueguang Ma, Shengyao Zhuang, Luyu Gao, Wenqian Ye, Yu Wang, Jamie Callan, Jimmy Lin

llms rag fine-tuning efficiency benchmarks data-engineering

One email a morning, machine-written

Related stories

TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology

TESSERA v2: Scaling Pixel-wise Earth Foundation Models

Cheap Probes Predict Expensive Training in 3D-CT Vision--Language Models

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget