arXiv AI July 16, 2026 By Qingyu Zhang, Qianhao Yuan, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun, Xiang Li, Ming Xu, Jiarui Li, Xiuyin Zhao

ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation

Read the original on arXiv AI →

arXiv:2607. 13124v1 Announce Type: cross Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Sep 10

Train Overcomplete, Deploy Compact: Scaling Recovery Capacity for Structured LLM Pruning

arXiv:2609.06974v1 Announce Type: cross Abstract: Large language models achieve strong performance across diverse tasks, but deployment remains costly because of memory, latency, and energy demands....

By Seungmin Oh, Donggeon Lee, Jongbin Ryu

llms efficiency

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv AI

Jun 2

Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning

arXiv:2605. 07804v3 Announce Type: replace-cross Abstract: On-policy distillation (OPD) leverages dense teacher rewards to enhance reasoning models.

By Zhicheng Yang, Zhijiang Guo, Yifan Song, Minrui Xu, Yongxin Wang, Yiwei Wang, Xiaodan Liang, Jing Tang

efficiency benchmarks

arXiv AI

Aug 5

Approximate Speculative Decoding

arXiv:2608. 03447v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel.

By Yuannuo Feng, Zegang Peng, Yuxin Xie, Yubing Ye, Yizhe Chen, Wenshuai Yao, Wenyong Zhou, Wang Kang

fine-tuning efficiency

arXiv AI

Aug 24

Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs

arXiv:2608.20953v1 Announce Type: cross Abstract: Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and...

By Bakbergen Ryskulov, Iker Garc\'ia-Ferrero, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Rom\'an Or\'us

llms efficiency benchmarks

arXiv AI

Aug 5

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

arXiv:2608. 03796v1 Announce Type: cross Abstract: Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model is usually recovered through knowledge distillation (KD).

By Bakbergen Ryskulov, Iker Garc\'ia-Ferrero, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Rom\'an Or\'us

llms efficiency benchmarks

One email a morning, machine-written

Related stories

Train Overcomplete, Deploy Compact: Scaling Recovery Capacity for Structured LLM Pruning

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning

Approximate Speculative Decoding

Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss