arXiv AI
Student-Guided Teacher Distillation for Efficient LLM Task Routing: Positioning Against Jev-Style System-1 Classifiers
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
6d ago
Decoupled and Distilled: Task-Adaptive LoRA-Teachers with Ensemble Knowledge Transfer for Few-Shot Class-Incremental Learning
The paper introduces TALON, a Task‑Adaptive LoRA‑Teacher framework for Few‑Shot Class‑Incremental Learning. TALON assigns a dedicated LoRA‑Teacher to each incremental task, then distills the frozen teachers into a single LoRA‑Student via Ensemble Knowledge Transfer, using a semantic‑guided weighting scheme to reduce forgetting and overfitting. Experiments on four FSCIL benchmarks show that TALON matches or surpasses state‑of‑the‑art accuracy while using up to 33× fewer deployment parameters and cutting inference time by 41.7%.
arXiv AI
6d ago
Distilling Directional Verification
arXiv:2610.00997v1 Announce Type: cross Abstract: Knowledge distillation aims to transfer the factual knowledge of large language models to smaller models for efficient deployment. Yet a teacher may...
arXiv Machine Learning
Jul 22
A Classifier That Teaches Itself: Self-Improving, Frozen-gate Training (SIFT) for Dynamic Document Classification
arXiv:2607. 18358v1 Announce Type: cross Abstract: Document classification is a solved problem in the laboratory and an unsolved one in the enterprise.
arXiv AI
6d ago
Distilling LLM Reasoning into Graph of Concept Predictors
The paper introduces Graph of Concept Predictors (GCP), a reasoning-aware active distillation framework that captures a large language model’s intermediate reasoning as a directed acyclic graph of concepts and mirrors this structure in a smaller student model. GCP improves sample efficiency by using a graph-aware acquisition strategy that weighs concept uncertainty, gradient diversity, and node centrality, and enhances training stability through targeted sub‑module retraining that updates only the most influential concept predictors. Experiments on eight NLP classification benchmarks show that GCP achieves better performance under limited annotation budgets while providing more interpretable and controllable training dynamics.
arXiv Computer Vision
Sep 14
RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs
arXiv:2609.12552v1 Announce Type: new Abstract: Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left t...
Hugging Face Trending Papers
Sep 2
SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking
SHELF is a Python system that creates controlled benchmark data and evaluation tasks for libraries and archives, using labelled taxonomies, writing specifications, and a generation budget. It generates 62,899 model-written documents based on Library of Congress vocabularies and supports tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. The release compares various methods—including TF, TF-IDF, BM25, popular encoders, and zero-shot decoders—showing that sparse methods remain competitive on classification and that SHELF can vary bibliographic facets independently while generating new, verifiably unseen documents.