arXiv Machine Learning
PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 3
Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
arXiv:2607. 01480v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifier and updates the policy from that episode-level signal.
arXiv AI
2d ago
Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models
arXiv:2607.22629v3 Announce Type: replace Abstract: Large Reasoning Models produce long, explicit chains of intermediate steps before generating a final answer at inference time. These intermediate t...
arXiv AI
Jul 3
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
arXiv:2607. 02234v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-level supervision on the student's own generated trajectories.
arXiv AI
Jun 8
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).
arXiv AI
Aug 10
MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents
arXiv:2608. 07068v1 Announce Type: new Abstract: Long-horizon agents accumulate growing contexts during interaction, impairing performance and stability.
arXiv Machine Learning
Jul 22
H$^2$SD: Hybrid Hindsight Self-Distillation
arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.