arXiv Machine Learning
Aug 14
Latent On-Policy Self-Distillation
arXiv:2608. 13040v1 Announce Type: new Abstract: Enabling agents to learn from experience and internalize it into their policy has become a central problem in self-evolving AI.
Self-Evolving Agents through Privileged Retrospective Distillation
Related stories
Hugging Face Trending Papers
Aug 13
Latent On-Policy Self-Distillation
Enabling agents to learn from experience and internalize it into their policy has become a central problem in self-evolving AI. On-policy self-distillation (OPSD) offers an effective pathway by using a privileged self-teacher to provide dense supervision on the student's own trajectories; however, existing methods still rely heavily on designer-specified privileged artifacts (e.
arXiv Machine Learning
Oct 3
Learning from the Near Future: Temporal Self-Distillation for RLVR
The paper introduces temporal self‑distillation for reinforcement learning with verifiable rewards (RLVR), proposing that a policy can learn from a stronger future checkpoint of itself. Two methods—Near‑Future Policy Optimization (NPO) and Near‑Future Policy Distillation (NPD)—use verified future‑self trajectories and token‑level transfer, respectively, while AutoNPO adaptively selects the optimal future checkpoint. Experiments on eight image‑text benchmarks show that near‑future teachers yield higher performance than far‑future ones, indicating that the balance between new capability and learner compatibility is key.
arXiv Machine Learning
Sep 30
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
arXiv:2609.37915v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies...
arXiv Computation and Language
Sep 28
Recursive Self-Improvement via On-Policy Distillation for Reasoning
The paper introduces a recursive self-improvement framework for language models that replaces an external teacher with a frozen copy of the student, enabling dynamic co-evolution (DCE) and self-refined concise learning (SRCL). DCE allows the privileged teacher to evolve alongside the student, while SRCL trains on shorter, verified rewrites to reduce verbosity. Experiments show that the combined DCE+SRCL approach outperforms traditional on‑policy self‑distillation across multiple model sizes and math benchmarks, achieving significant accuracy gains and shorter outputs.
Hugging Face Trending Papers
Aug 5
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated. On-Policy Self-Distillation (OPSD) addresses this by re-scoring generated tokens under a privileged replay view to obtain dense, token-level supervision.
arXiv AI
Sep 30
Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
arXiv:2609.37132v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) improves large language models by letting a self-teacher with privileged information provide dense token-level super...
arXiv AI
Aug 7
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.
arXiv Machine Learning
Jul 31
Group-Reflective Self-Distillation for Agentic Reinforcement Learning
arXiv:2607. 28076v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents.
arXiv AI
Aug 6
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
arXiv:2608. 04788v1 Announce Type: cross Abstract: Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated.
arXiv AI
Oct 7
Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
The paper introduces Self‑Retrospection Distillation (SRD), a method that uses post‑hoc trajectories to supervise foresight predictions made before an agent acts. SRD distills information from completed rollouts into trajectory‑blind predictions, enabling the agent to anticipate useful actions and avoid pitfalls without requiring explicit foresight at inference time. Experiments on ten tool‑integrated reasoning and long‑horizon tasks show that SRD improves performance by up to 24.2 percentage points, especially when reward signals are scarce or uniform across rollouts.
arXiv AI
Aug 11
SR-OPSD: Self-Referenced On-Policy Self-Distillation
arXiv:2608. 09745v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on trajectories generated by the policy to be optimized, providing a useful complement to reinforcement learning with sparse outcome rewards.