arXiv AI

When Context Returns: Toward Robust Internalization in On-Policy Distillation

arXiv:2606. 11627v1 Announce Type: cross Abstract: Recent work has shown that on-policy distillation can internalize privileged context, such as system prompts or task hints, into a student model so that the context is no longer needed at inference time.

arXiv Machine Learning

Aug 11

Adaptive Supervised Anchoring for On-Policy Self-Distillation

arXiv:2608. 07935v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student.

By Meilin Yang (Renmin University of China, Beijing, China), Zixuan Ding (Renmin University of China, Beijing, China), Jianhao Nie (Renmin University of China, Beijing, China), Weite Zhang (Renmin University of China, Beijing, China), Yuxin Zhang (Renmin University of China, Beijing, China), Zhiming Shao (Renmin University of China, Beijing, China), Li Yu (Renmin University of China, Beijing, China), Zhe Fu (Renmin University of China, Beijing, China)

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 4

Rethinking Continual Experience Internalization for Self-Evolving LLM Agents

arXiv:2606. 04703v1 Announce Type: cross Abstract: Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large language models (LLMs).

By Jingwen Chen, Wenkai Yang, Shengda Fan, Wenbo Nie, Chenxing Sun, Shaodong Zheng, Yangen Hu, Lu Pan, Ke Zeng, Yankai Lin

llms agents efficiency

arXiv AI

Jun 9

Trajectory-Refined Distillation

arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.

By Li Jiang, Haoran Xu, Yichuan Ding, Amy Zhang

llms efficiency benchmarks

arXiv Machine Learning

Jul 31

Flux-OPD: On-Policy Distillation with Evolving Contexts

arXiv:2607. 28022v1 Announce Type: new Abstract: Large language model training in open-ended domains lacks verifiable rewards, making task preferences difficult to formalize as effective supervision.

By Yuran Wang, Zekun Wang, Bohan Zeng, Ruixu Zhang, Wenxuan Liu, Liu Yang, Yifan Dai, Yang Shi, Bozhou Li, Chengzhuo Tong, Daili Hua, Yuanxing Zhang, Wentao Zhang

llms efficiency

arXiv AI

Aug 5

When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation

arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.

By Yinuo Jiang, Yongjie Ye, Zhou Tao, Xiang Zhuang, Qiang Zhang, Huajun Chen, Tiankai Li

llms efficiency multimodal

arXiv AI

Jun 2

Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

arXiv:2606. 00105v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on vision-language tasks, but they may also memorize and expose sensitive or restricted knowledge, raising concerns about privacy and broader safety risks.

By Junkai Chen, Yuhao He, Junxiang You, Ruiqi Liu, Chenyu Wang, Shu Wu

llms efficiency multimodal safety

arXiv Machine Learning

Jul 22

One Student, Many Teachers: Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context

arXiv:2607. 18293v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) teaches large language models new skills through a teacher that shares the student's backbone and supervises its own rollouts.

By Yingzi Ma, Zichen Zhu, Ming Jiang, Chaowei Xiao

llms agents fine-tuning efficiency benchmarks

arXiv AI

Jul 3

DemoPSD: Disagreement-Modulated Policy Self-Distillation

arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.

By Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song

llms efficiency benchmarks

arXiv AI

Jul 7

Rethinking On-Policy Self-Distillation for Thinking Models

arXiv:2607. 05184v1 Announce Type: new Abstract: Self-distillation is a promising recipe for self-improvement in language models.

By Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, Sanjeev Arora

llms efficiency

Hugging Face Trending Papers

Jul 30

Flux-OPD: On-Policy Distillation with Evolving Contexts

Large language model training in open-ended domains lacks verifiable rewards, making task preferences difficult to formalize as effective supervision. Contexts can convey such preferences, yet provide little additional supervision once distilled into the student, motivating contexts that evolve with student performance.

llms efficiency

arXiv AI

Aug 10

MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents

arXiv:2608. 07068v1 Announce Type: new Abstract: Long-horizon agents accumulate growing contexts during interaction, impairing performance and stability.

By Zhiyuan Liu, Tinghong Ye, Chenghao Liu, Yizhuo Li, Songfang Huang

agents reinforcement-learning efficiency safety

Hugging Face Trending Papers

Aug 4

When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation

On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve this process by prioritizing signals that are confident, informative, or learnable.

llms efficiency multimodal

Related stories