arXiv AI July 10, 2026 By Bowen Yu, Maolin Wang, Sheng Zhang, Binhao Wang, Yi Wen, Jingtong Gao, Bowen Liu, Zimo Zhao, Wanyu Wang, Xiangyu Zhao

Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO

Read the original on arXiv AI →

arXiv:2602. 17686v4 Announce Type: replace-cross Abstract: Distilling Chain-of-Thought (CoT) reasoning from large language models into compact student models presents a fundamental challenge: teacher rationales are often too verbose for smaller models to faithfully reproduce.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 14

BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

arXiv:2602. 17686v5 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) reasoning has significantly improved LLMs' mathematical problem-solving capabilities, but distilling such capabilities into smaller models remains challenging due to the capacity mismatch between verbose teachers and compact students.

By Bowen Yu, Sheng Zhang, Binhao Wang, Yi Wen, Jingtong Gao, Bowen Liu, Zimo Zhao, Shanshan Ye, Wanyu Wang, Maolin Wang, Xiangyu Zhao

llms reinforcement-learning efficiency safety

arXiv AI

2d ago

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

arXiv:2607.22629v3 Announce Type: replace Abstract: Large Reasoning Models produce long, explicit chains of intermediate steps before generating a final answer at inference time. These intermediate t...

By Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri, Subbarao Kambhampati

llms fine-tuning efficiency benchmarks

arXiv AI

3d ago

STEPS: Selective On-Policy Self-Distillation for Reasoning

arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own r...

By Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Lan-Zhe Guo

reinforcement-learning efficiency benchmarks

arXiv Computation and Language

Sep 15

Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR

arXiv:2609.13997v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has shown remarkable success in improving the mathematical reasoning of large language models. Ye...

By Yukang Zhu, Zhen Han

llms reinforcement-learning benchmarks

arXiv AI

Jun 19

Reinforcement-aware Knowledge Distillation for LLM Reasoning

arXiv:2602. 22495v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such models motivates distillation into smaller students.

By Zhaoyang Zhang, Shuli Jiang, Yantao Shen, Yuting Zhang, Dhananjay Ram, Shuo Yang, Zhuowen Tu, Wei Xia, Stefano Soatto

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Aug 11

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation

arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.

By Jaehun Jung, Hyunwoo Kim, Brandon Cui, Ximing Lu, David Acuna, Prithviraj Ammanabrolu, Yejin Choi

llms fine-tuning efficiency multimodal benchmarks

One email a morning, machine-written

Related stories

BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

STEPS: Selective On-Policy Self-Distillation for Reasoning

Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR

Reinforcement-aware Knowledge Distillation for LLM Reasoning

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation