arXiv AI October 23, 2026 By Shuo Yuan, Dacheng Li, Joseph E. Gonzalez

D-Skill: Distilling Teacher Knowledge through Agent Skills

D‑Skill is a framework that distills the behavioral advantage of a large language‑model teacher into an interpretable natural‑language skill document, using only paired rollouts and no weight updates. The skill is constructed offline from fixed trajectories, then provided to a smaller student model at inference time. Across benchmarks such as SpreadsheetBench‑Verified, ALFWorld, and chess puzzles, D‑Skill closes 75‑95% of the teacher‑student performance gap and reduces inference cost by 5.7×–8.3× compared to the teacher.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 5

SKILL-KD: Contrastive Skill Distillation for LLM Agents

arXiv:2607. 28048v2 Announce Type: replace Abstract: Skill-based prompting has become a practical mechanism for improving large language model (LLM) agents, yet existing skill acquisition methods often treat skills as experience summaries, memory entries, or direct summaries of successful demonstrations.

By Qiming Shi, Yibo Dou, Jiawen Zhu, Yulong Tao, Linbo Jin, Zhaolu Kang, Yunfan Zhou, Di Weng

llms agents efficiency benchmarks

arXiv AI

Aug 5

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

arXiv:2608. 03206v1 Announce Type: cross Abstract: Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS).

By Unggi Lee, Sookbun Lee, Yeil Jeong, Eunjoo Lee, Minchul Shin, Hoilym Kwon

llms agents benchmarks

arXiv AI

Oct 5

Harness-Aware Distillation for Small Language Model Agents

The paper introduces Harness-Aware Distillation (HAD), a method for training smaller language model agents that preserves the surrounding harness—software managing context, tools, and feedback—while focusing distillation on the teacher’s contributions beyond the harness. HAD combines an action preference that contrasts teacher actions with and without harness information, and a validity check that filters out contradictory preference pairs. Experiments on long-horizon agent benchmarks show that HAD outperforms standard on‑policy distillation, reducing unproductive loops and improving error recovery without requiring task rewards or future information.

By Moonseok Choi, Taehong Moon, Giung Nam, Juho Lee

llms agents efficiency benchmarks

arXiv Machine Learning

Oct 15

Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

arXiv:2610.19439v1 Announce Type: new Abstract: While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substanti...

By Yuanpu Cao, Saket Sathe, Hanyu Wang, Ziyi Yin, Fenglong Ma, Jinghui Chen

llms agents fine-tuning efficiency benchmarks

arXiv Computation and Language

Sep 24

SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving

arXiv:2609.27717v1 Announce Type: new Abstract: Human-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather...

By Zhilong Ge, Yuting Shao, Yutao Yang, Yuxuan Cai, Jie Zhou, Kai Chen, Bo Zhang, Qin Chen, Liang He

llms agents reinforcement-learning fine-tuning

arXiv AI

Jul 1

Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition

arXiv:2606. 31048v1 Announce Type: cross Abstract: This paper investigates knowledge distillation from a large reasoning model (DeepSeek-R1) to a compact student model (Qwen2.

By Gaurab Baral, Aaditya Khanal, Yangyang Tao, Junxiu Zhou

agents fine-tuning efficiency benchmarks

One email a morning, machine-written

Related stories

SKILL-KD: Contrastive Skill Distillation for LLM Agents

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

Harness-Aware Distillation for Small Language Model Agents

Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving

Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition