arXiv Computation and Language October 9, 2026 By Yuhan Liu, Xiyao Ma, Zhongkai Sun, Xu Han, Chengyuan Ma, Benjamin Z. Yao, Chenlei Guo

Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution

Read the original on arXiv Computation and Language →

The paper introduces SGUID, a method that selects a compact subset of skills for distillation in large language models. It shows that fewer than 25% of retrieved skills are useful for on‑policy distillation and that distilling only the selected skills can match or surpass full‑bank distillation while using up to 11× fewer skills. SGUID also enables stable model‑skill co‑evolution by iteratively curating new skill banks from updated model rollouts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computer Vision

4d ago

SkillCycle: Co-Evolving Agent Policies and Skill Banks

arXiv:2610.09430v1 Announce Type: new Abstract: Internalizing external skills changes a language agent's capabilities and, with them, the value of its remaining guidance: rules can become redundant,...

By Ling Li, Qiuyu Shen, Zheng Jiang, Qinwei Ma, Yuxuan Liu, Zhidong Deng

agents efficiency benchmarks

arXiv AI

Aug 11

Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

arXiv:2608. 09826v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63.

By Yubo Jiang, Fengying Xie, Zhiguo Jiang, Haopeng Zhang

reinforcement-learning efficiency benchmarks

Hugging Face Trending Papers

Aug 10

Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63. 0-68.

reinforcement-learning efficiency benchmarks

arXiv Computation and Language

Sep 28

Recursive Self-Improvement via On-Policy Distillation for Reasoning

The paper introduces a recursive self-improvement framework for language models that replaces an external teacher with a frozen copy of the student, enabling dynamic co-evolution (DCE) and self-refined concise learning (SRCL). DCE allows the privileged teacher to evolve alongside the student, while SRCL trains on shorter, verified rewrites to reduce verbosity. Experiments show that the combined DCE+SRCL approach outperforms traditional on‑policy self‑distillation across multiple model sizes and math benchmarks, achieving significant accuracy gains and shorter outputs.

By Shangjian Yin, Zehao Zhao, Kavosh Asadi, Rui Liu, Yuchen Lu, Shike Mei, Hang Cui, Luke Simon, Zhouxing Shi, Hamed Firooz

efficiency benchmarks

arXiv AI

Oct 1

STEPS: Selective On-Policy Self-Distillation for Reasoning

arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own r...

By Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Lan-Zhe Guo

reinforcement-learning efficiency benchmarks

arXiv AI

Aug 26

Skill-Conditioned Gated Self-Distillation for LLM Reasoning

arXiv:2605.28791v2 Announce Type: replace-cross Abstract: On-policy self-distillation (SD) improves LLM reasoning by using teacher-side privileged information (PI) to turn sparse verifier outcomes in...

By Jiazhen Huang, Xiao Chen, Xiao Luo, Yong Dai, Senkang Hu, Yuzhi Zhao

llms efficiency benchmarks

One email a morning, machine-written

Related stories

SkillCycle: Co-Evolving Agent Policies and Skill Banks

Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

Recursive Self-Improvement via On-Policy Distillation for Reasoning

STEPS: Selective On-Policy Self-Distillation for Reasoning

Skill-Conditioned Gated Self-Distillation for LLM Reasoning