arXiv AI August 6, 2026 By Sarthak Harne, Chinmay Karkar, Yash Pandya, Ahmed Awadallah, Akshay Nambi

Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation

arXiv:2608. 04794v1 Announce Type: new Abstract: Self-distillation (SD) has emerged as a compute-efficient alternative to reinforcement learning with verifiable rewards: a self-teacher, conditioned on privileged information (PI) about the answer such as a reference solution, supplies dense per-token supervision to a student that never sees it.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 22

H$^2$SD: Hybrid Hindsight Self-Distillation

arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.

By Qiye Cai, Yichuan Ma, Linyang Li, Peiji Li, Yongkang Chen, Qipeng Guo, Yicheng Zou, Tao Gui, Xiaocheng Feng, Bing Qin

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jul 3

DemoPSD: Disagreement-Modulated Policy Self-Distillation

arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.

By Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song

llms efficiency benchmarks

arXiv AI

Aug 11

SR-OPSD: Self-Referenced On-Policy Self-Distillation

arXiv:2608. 09745v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on trajectories generated by the policy to be optimized, providing a useful complement to reinforcement learning with sparse outcome rewards.

By Zhuo Sun, Entong Li, Yanlong Zhao, Xiaoyuan Cheng, Wenxuan Yuan, Kaiyu Li, Che Liu, Huihang Liu, Harrison Bo Hua Zhu, Li Zeng

llms reinforcement-learning efficiency benchmarks

arXiv AI

Aug 11

Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation

arXiv:2608. 08176v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) improves the reasoning abilities of LLMs by internalizing privileged context into model parameters through self-distillation.

By Yongkang Yang, Zhezheng Hao, Hong Zhang, Yi Liu, Xiankun Lin, Wence Ji, Fanjunduo Wei, Jiarui Yu, Qiang Lin, Xiaoyun Liang, Hande Dong

llms efficiency benchmarks

arXiv AI

Jun 10

Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning

arXiv:2605. 11458v3 Announce Type: replace Abstract: On-policy self-distillation has become a strong recipe for LLM reasoning, where a privileged teacher supervises the student's own rollouts while conditioning on the reference solution.

By Zihao Han, Tiangang Zhang, Huaibin Wang, Yilun Sun

llms reinforcement-learning efficiency

arXiv AI

Jun 2

CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO

arXiv:2606. 00172v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), especially Group Relative Policy Optimization (GRPO), has been widely used to improve reasoning in large language models.

By Yang Li, Gongle Xue, Yijia Guo, Yuheng Yuan, Liwen Hu, Lei Ma

llms reinforcement-learning efficiency

Related stories

H$^2$SD: Hybrid Hindsight Self-Distillation

DemoPSD: Disagreement-Modulated Policy Self-Distillation

SR-OPSD: Self-Referenced On-Policy Self-Distillation

Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation

Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning

CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO