The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
LoopOPD introduces a cross-loop on-policy distillation framework that uses a frozen terminal loop policy as a compute-privileged teacher for intermediate loop students, providing dense supervision without external teachers. Dynamic LoopOPD (D-LoopOPD) further refreshes the terminal loop teacher as model parameters update, enabling recurrent self‑improvement. Experiments on Ouro‑Thinking models show that LoopOPD enhances mathematical reasoning and that D‑LoopOPD yields additional gains, with the models also improving on general reasoning and code generation benchmarks.
arXiv:2609.36636v1 Announce Type: new Abstract: Looped language models (LoopLMs) increase computational depth through parameter sharing, offering a path to scale inference computation without adding...
arXiv:2603.19714v2 Announce Type: replace Abstract: Looped language models (LoopLMs) perform iterative latent computation to refine internal representations, offering an alternative to explicit chain...
arXiv:2610.18162v1 Announce Type: cross Abstract: On-policy self-distillation improves a model using guidance from the model itself, but it still requires a teacher that provides more useful predicti...
SIPO (Self‑Instructing Policy Optimization) unifies reinforcement learning with on‑policy self‑distillation by using a contrastive self‑teacher to generate token‑level credit signals. The method samples multiple rollouts per prompt, pairs each with a reference answer and its mistakes, and uses the difference in teacher log‑probabilities to provide dense feedback while still respecting the overall task reward. Experiments on reasoning and code‑generation benchmarks show that SIPO outperforms both RLVR and OPSD baselines without requiring an external teacher or extra generation steps.
SOD: Step-wise On-policy Distillation for Small Language Model Agents proposes a new framework that adaptively reweights distillation strength at each reasoning step based on step-level divergence. This approach mitigates cascading errors in tool-integrated reasoning by attenuating misleading teacher signals in high-divergence regions while preserving dense guidance where student and teacher align. Experiments on math, science, and code benchmarks show up to 20.86% improvement over the second-best baseline, with a 0.6B student scoring 26.13% on AIME 2025.