arXiv AI July 7, 2026 By Phuong Tuan Dat, Qi Li, Xinchao Wang

dOPSD: On-Policy Self-Distillation for Diffusion Language Models

arXiv:2607. 04428v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 21

Trace-Based On-Policy Distillation for Masked Diffusion Language Models

arXiv:2607. 16872v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation.

By Haolin Ren, Ziyang Huang, Chenhao Yuan, Jun Zhao, Kang Liu

llms diffusion reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 22

H$^2$SD: Hybrid Hindsight Self-Distillation

arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.

By Qiye Cai, Yichuan Ma, Linyang Li, Peiji Li, Yongkang Chen, Qipeng Guo, Yicheng Zou, Tao Gui, Xiaocheng Feng, Bing Qin

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jul 3

DemoPSD: Disagreement-Modulated Policy Self-Distillation

arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.

By Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song

llms efficiency benchmarks

arXiv AI

Aug 6

Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation

arXiv:2608. 04794v1 Announce Type: new Abstract: Self-distillation (SD) has emerged as a compute-efficient alternative to reinforcement learning with verifiable rewards: a self-teacher, conditioned on privileged information (PI) about the answer such as a reference solution, supplies dense per-token supervision to a student that never sees it.

By Sarthak Harne, Chinmay Karkar, Yash Pandya, Ahmed Awadallah, Akshay Nambi

agents nlp reinforcement-learning efficiency safety

arXiv AI

Jul 3

Purified OPSD: On-Policy Self-Distillation Without Losing How to Think

arXiv:2607. 02234v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-level supervision on the student's own generated trajectories.

By Zhanming Shen, Jintao Tong, Shaotian Yan, Chen Shen, Hao Chen, Wentao Ye, Xiaomeng Hu, Rui Miao, Haobo Wang, Junbo Zhao, Gang Chen, Jieping Ye

llms efficiency

arXiv AI

Aug 5

When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation

arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.

By Yinuo Jiang, Yongjie Ye, Zhou Tao, Xiang Zhuang, Qiang Zhang, Huajun Chen, Tiankai Li

llms efficiency multimodal

Related stories

Trace-Based On-Policy Distillation for Masked Diffusion Language Models

H$^2$SD: Hybrid Hindsight Self-Distillation

DemoPSD: Disagreement-Modulated Policy Self-Distillation

Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation

Purified OPSD: On-Policy Self-Distillation Without Losing How to Think

When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation