← Back to all news
arXiv AI By Seongjae Kang, Dong Bok Lee, Hyungjoon Jang, Dongseop Kim, Sung Ju Hwang

PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models

Read the original on arXiv AI →

The paper introduces ActiveKD, a framework that combines active learning with knowledge distillation by using large vision‑language models as teachers. It leverages the structured prediction bias of these models to design Probabilistic CoreSet (PCoreSet), a selection strategy that maximizes coverage in probability space rather than feature space. Experiments on 11 datasets show that ActiveKD improves all selection methods over a no‑distillation baseline, and PCoreSet consistently outperforms random sampling across multiple students and teachers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 7

What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection

The paper investigates data efficiency and selection in On‑Policy Distillation (OPD) for large language models. It shows that 1‑shot OPD—training on a single example—consistently improves performance, especially when the example is hard, and that longer chain‑of‑thought (CoT) paths drive the gains rather than token entropy. Based on these findings, the authors propose a simple hard‑example selection strategy that, using only eight carefully chosen hard examples, matches the performance of a 17,000‑example baseline across models from 1.5B to 7B parameters.

By Zhinan Hou, Jiaqi Zhang, Xunliang Cai, Keyou You
llmsefficiencysafety
More like this →