The paper investigates how much training data is required for on‑policy distillation (OPD) of foundation models. It shows that in two single‑teacher settings, only eight real prompts can match the performance of 17,000 problems, and that 64 self‑generated questions can replace full external datasets. In multi‑teacher scenarios, 1,000 generated questions achieve comparable results to 7,000 real examples, while fully implicit question generation works only in limited cases.
By Gengsheng Li, Mao Zheng, Mingyang Song, Jie Sun, Zeyuan Liu, Ruiqi Liu, Tianyu Yang, Qiyong Zhong, Haiyun Guo, Junfeng Fang, Shiming Xiang, Jinqiao Wang, Tat-Seng Chua
BoundaryEvo introduces a two‑stage data‑model co‑evolution framework for distilling long‑chain reasoning from large teacher models into small language models. First, Capability Priming distills a small set of teacher trajectories to give the student an initial reasoning baseline. Then, Capability‑Boundary Co‑Evolution iteratively selects questions the student has not mastered but the teacher can solve, updating the training data as the student’s capability grows. With only 1,000 examples, BoundaryEvo improves Qwen3.5‑4B accuracy from 62.9 % to 75.3 % on IMO‑AnswerBench and from 26.8 % to 46.0 % on AMO‑Bench, outperforming static and dynamic baselines across 11 benchmarks.
By Jiaming Su, Suorong Yang, Xudong Li, Tengchao Yang, Peng Ye
arXiv:2512. 21002v3 Announce Type: replace-cross Abstract: Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data.
By Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi, Hejian Sang, Shao Tang, Qingquan Song, Zhipeng Wang, Muhammad Abdul-Mageed
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
The paper investigates on‑policy distillation (OPD) of large language models when trained on a minimal amount of data, specifically a single query. It finds that one‑shot OPD continues to improve for hundreds of steps and recovers most of the performance gains achieved with full‑dataset training across various tasks and model families. The authors attribute this to the high state coverage achieved early in training and show that adding a few semantically distinct queries further boosts coverage and accuracy, eventually matching full‑data results. They conclude that OPD is data‑overfed but algorithm‑starved, highlighting the need to focus on step efficiency and the underlying mechanisms of its recent successes.
By Zixuan Fu, Bingxiang He, Yuxin Zuo, Haohuan Huang, Jinqian Zhang, Ruhang Xiao, Cheng Qian, Qinyu Luo, Huan-ang Gao, Yudong Wang, Zhiyuan Liu, Ning Ding, Chaojun Xiao
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang