← Back to all news
arXiv Machine Learning By Wenjin Liu, Chenxi Wang, Jiapu Wang, Zhe Cui, Anh Tuan Luu, Haoran Luo

CataOPD: Catalytic On-Policy Distillation for Large Language Model Reasoning

Read the original on arXiv Machine Learning →

CataOPD introduces a new framework for improving large language model reasoning by combining reinforcement learning and on‑policy distillation. The method treats the teacher as a catalyst that expands the student’s reachability, using Self‑Rescue Routing to find correct trajectories through additional on‑policy sampling and Catalytic‑Guided Self‑Resolution to elicit verified student trajectories. Barrier‑Weighted Internalization further focuses updates on decisive tokens, leading to better performance on unseen problems and improved out‑of‑distribution generalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 22

H$^2$SD: Hybrid Hindsight Self-Distillation

arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.

By Qiye Cai, Yichuan Ma, Linyang Li, Peiji Li, Yongkang Chen, Qipeng Guo, Yicheng Zou, Tao Gui, Xiaocheng Feng, Bing Qin
llmsreinforcement-learningefficiencybenchmarks
More like this →
arXiv AI
Aug 19

SOD: Step-wise On-policy Distillation for Small Language Model Agents

SOD: Step-wise On-policy Distillation for Small Language Model Agents proposes a new framework that adaptively reweights distillation strength at each reasoning step based on step-level divergence. This approach mitigates cascading errors in tool-integrated reasoning by attenuating misleading teacher signals in high-divergence regions while preserving dense guidance where student and teacher align. Experiments on math, science, and code benchmarks show up to 20.86% improvement over the second-best baseline, with a 0.6B student scoring 26.13% on AIME 2025.

By Qiyong Zhong, Mao Zheng, Mingyang Song, Xin Lin, Jie Sun, Houcheng Jiang, Xiang Wang, Junfeng Fang
llmsagentsreinforcement-learningefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Jun 17

Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation

Self-distillation improves reasoning in large language models by using the model's own rollouts as training signal, typically through implicit logit-level alignment that minimizes KL divergence toward a privileged target distribution. However, because this supervision is generated via uncontrolled sampling, it provides no diagnostic insight into the model's specific errors or corrective guidance for its individual failure patterns.

llmsreinforcement-learningefficiencysafety
More like this →