SPT: Skills as Pre-Training Data for Agentic Language Models

The paper introduces Skill Pre-Training (SPT), a mid‑training approach that uses public skill packages as data for agentic language models. By applying causal language modeling to a collection called SkillCorpus and employing a Reference Insert strategy to keep file relationships intact, SPT improves agentic performance across various model sizes while largely preserving general capabilities. Experiments also show that mixing skill data with general corpora yields further benefits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• agents