arXiv AI June 19, 2026 By Zhaoyang Zhang, Shuli Jiang, Yantao Shen, Yuting Zhang, Dhananjay Ram, Shuo Yang, Zhuowen Tu, Wei Xia, Stefano Soatto

Reinforcement-aware Knowledge Distillation for LLM Reasoning

Read the original on arXiv AI →

arXiv:2602. 22495v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such models motivates distillation into smaller students.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 19

A Survey of On-Policy Distillation for Large Language Models

arXiv:2604. 00626v4 Announce Type: replace Abstract: As Large Language Models continue to grow in both capability and cost, transferring frontier capabilities into smaller, deployable students has become an important engineering problem, and knowledge distillation remains a common technique for this transfer.

By Mingyang Song, Mao Zheng

llms agents reinforcement-learning efficiency safety

arXiv AI

Jul 21

Distilled Reinforcement Learning for LLM Post-training

arXiv:2607. 17247v1 Announce Type: cross Abstract: Large language model (LLM) post-training is essential for improving reasoning, adaptation, and alignment.

By Chen Wang, Zhaochun Li, Jionghao Bai, Yining Zhang, Hexuan Deng, Ge Lan, Yue Wang

llms reinforcement-learning efficiency safety

arXiv AI

Jul 23

REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

arXiv:2607. 19450v1 Announce Type: cross Abstract: Large-scale online reinforcement learning (RL) is the predominant means of eliciting advanced abilities including long-term reasoning and agentic tool use in large language models (LLMs).

By Yunjie Chen, Xiaoxin Chen, Fang Wang

llms agents reinforcement-learning efficiency

Hugging Face Trending Papers

Jul 21

REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

Large-scale online reinforcement learning (RL) is the predominant means of eliciting advanced abilities including long-term reasoning and agentic tool use in large language models (LLMs). However, continuing to scale it across vast task domains of interest remains challenging in both computational infrastructure and cost, especially when considering RL as merely a one-off learning stage.

llms agents reinforcement-learning efficiency

arXiv AI

Jul 24

Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs

arXiv:2509. 14257v3 Announce Type: replace-cross Abstract: Large Language Model agents achieve strong performance on multi-step reasoning and tool-use tasks, but their impressive capabilities typically rely on extremely large backbones.

By Yuanjie Lyu, Chengyu Wang, Jun Huang, Tong Xu

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

3d ago

STEPS: Selective On-Policy Self-Distillation for Reasoning

arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own r...

By Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Lan-Zhe Guo

reinforcement-learning efficiency benchmarks

One email a morning, machine-written

Related stories

A Survey of On-Policy Distillation for Large Language Models

Distilled Reinforcement Learning for LLM Post-training

REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs

STEPS: Selective On-Policy Self-Distillation for Reasoning