arXiv AI
Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 7
Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification
arXiv:2410.21582v4 Announce Type: replace-cross Abstract: Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of mainta...
arXiv AI
Aug 20
From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model
arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.
arXiv AI
Jul 17
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
arXiv:2607. 14703v1 Announce Type: cross Abstract: Multiple instance learning (MIL) has become the main paradigm for whole-slide image (WSI) analysis in computational pathology.
arXiv Computer Vision
Oct 15
Enabling Validation for Robust Few-Shot Recognition
arXiv:2506.04713v4 Announce Type: replace Abstract: A key challenge in Few-Shot Recognition (FSR) is the scarcity of labeled task-specific data for training. Contemporary methods mitigate this challe...
arXiv AI
Sep 10
Precision at Scale: Domain-Specific Datasets On-Demand
arXiv:2407.03463v2 Announce Type: replace-cross Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets fo...
arXiv AI
2d ago
PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
The paper introduces ActiveKD, a framework that combines active learning with knowledge distillation by using large vision‑language models as teachers. It leverages the structured prediction bias of these models to design Probabilistic CoreSet (PCoreSet), a selection strategy that maximizes coverage in probability space rather than feature space. Experiments on 11 datasets show that ActiveKD improves all selection methods over a no‑distillation baseline, and PCoreSet consistently outperforms random sampling across multiple students and teachers.