arXiv AI June 2, 2026 By Junseok Lee, Nahun Kim, Sangyong Lee, Chang-Jae Chun

ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition

arXiv:2601. 19919v2 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Aug 25

SelFusion: Self-distillation for Diffusion Language Models

arXiv:2608.22898v1 Announce Type: new Abstract: Diffusion language models (DLMs) alleviate the inherent latency bottleneck of autoregressive (AR) large language models (LLMs), but their degraded gene...

By Hyeongsoo Lim, Jinyoung Kim, Eunseo Seo, Minho Jang, Jiwon Yoon

llms diffusion efficiency

arXiv AI

2d ago

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

arXiv:2607.22629v3 Announce Type: replace Abstract: Large Reasoning Models produce long, explicit chains of intermediate steps before generating a final answer at inference time. These intermediate t...

By Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri, Subbarao Kambhampati

llms fine-tuning efficiency benchmarks

arXiv AI

Aug 11

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation

arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.

By Jaehun Jung, Hyunwoo Kim, Brandon Cui, Ximing Lu, David Acuna, Prithviraj Ammanabrolu, Yejin Choi

llms fine-tuning efficiency multimodal benchmarks

arXiv AI

Jun 19

Wisdom of Committee: Diverse Distillation from Large Foundation Models and Domain Experts

arXiv:2402. 14035v4 Announce Type: replace-cross Abstract: Knowledge distillation from foundation models to compact domain models is challenging due to substantial gaps in capacity, architecture, and modality.

By Zichang Liu, Qingyun Liu, Yuening Li, Liang Liu, Anshumali Shrivastava, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao

llms efficiency

arXiv Computation and Language

4d ago

Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models

arXiv:2609.36734v1 Announce Type: new Abstract: Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implic...

By Ayan Sengupta, Vaibhav Seth, Tanmoy Chakraborty

llms efficiency benchmarks safety

arXiv AI

Jun 24

WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models

arXiv:2604. 08558v2 Announce Type: replace-cross Abstract: Recent decoder-only autoregressive text-to-speech (AR-TTS) models produce high-fidelity speech, but their memory and compute costs scale quadratically with sequence length due to full self-attention.

By Hanna Lee, Tan Dat Nguyen, Jaehoon Kang, Kyuhong Shim

fine-tuning efficiency multimodal

One email a morning, machine-written

Related stories

SelFusion: Self-distillation for Diffusion Language Models

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation

Wisdom of Committee: Diverse Distillation from Large Foundation Models and Domain Experts

Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models

WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models