arXiv AI

Representation Affects Retrieval: A Case Study of Skill Discovery and Routing in a Multimodal Agent Harness

The paper examines how skill representations influence selection in a multimodal video agent harness called Tinycloud. It compares two types of skill representations—tool-skills and workflow-skills—and two prompt surfaces—full inlined bodies and one-line listings—across three exposure regimes. The study finds that full autoload exposure consistently selects the correct skill, while partial exposure can cause lexical competition that misroutes tasks, highlighting that in-prompt exposure is not always beneficial.

arXiv AI

Sep 28

SkillFlow: Scalable and Efficient Agent Skill Retrieval System

SkillFlow is an open, multi-stage retrieval system that helps AI agents selectively load relevant skills from a large library of community-contributed SKILL.md definitions. The pipeline uses dense retrieval, two rounds of cross-encoder reranking, and LLM-based selection to balance recall and precision. Evaluations on SkillsBench and Terminal-Bench show that SkillFlow improves performance when high-quality skills are available, but retrieval alone does not help if the corpus lacks executable skills for the target domain.

By Fangzhou Li, Pagkratios Tagkopoulos, Ilias Tagkopoulos

llms rag agents benchmarks

arXiv AI

Jun 9

Skill Retrieval Augmentation for Agentic AI

arXiv:2604. 24594v3 Announce Type: replace-cross Abstract: As large language models (LLMs) evolve into agentic problem solvers, they increasingly rely on external, reusable skills to handle tasks beyond their native parametric capabilities.

By Weihang Su, Jianming Long, Qingyao Ai, Qiaozhi He, Yichen Tang, Changyue Wang, Yiteng Tu, Yingbo Wang, Yiqun Liu

llms agents benchmarks

arXiv AI

Sep 15

The Router Within: Eliciting Native Skill Routing from a Frozen LLM

arXiv:2609.15982v1 Announce Type: cross Abstract: Skills extend an LLM agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. Deployed harnesses route by pre...

By Ruishuo Chen, Xun Wang, Yu Chen, Zhuoran Li, Longbo Huang

llms agents benchmarks

arXiv AI

Sep 1

SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents

arXiv:2605.05726v2 Announce Type: replace Abstract: As LLM agents are increasingly deployed with large libraries of reusable skills, selecting the right skill for a user request has become a critical...

By Hongcheol Cho, Ryangkyung Kang, Youngeun Kim

llms rag agents fine-tuning benchmarks

arXiv Computation and Language

Aug 21

SPyCE: Skill-Policy Co-evolution for Multimodal Agents

arXiv:2607. 13854v2 Announce Type: replace Abstract: Multimodal agents that think with images iteratively manipulate visual evidence and invoke tools across many steps.

By Ru Zhang, Weijie Qiu

agents reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jul 21

SkillRouter: Skill Routing for LLM Agents at Scale

arXiv:2603. 22455v5 Announce Type: replace Abstract: Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks.

By YanZhao Zheng, ZhenTao Zhang, Chao Ma, YuanQiang Yu, JiHuai Zhu, Yong Wu, Tianze Xu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu

llms rag agents benchmarks

arXiv AI

Jul 22

SkillSight: Seeing Through Shared Descriptions for Accurate Skill Retrieval

arXiv:2607. 18785v1 Announce Type: new Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.

By Jinying Xiao, Bin Ji, Shasha Li, Xiaodong Liu, Ma Jun, Jiacheng Jie, Chao Wang, Nyima Tashi, Jie Yu

llms agents safety

arXiv AI

Aug 11

From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agents

arXiv:2608. 09168v1 Announce Type: new Abstract: Agent skills are increasingly used to equip large language model (LLM) agents with reusable procedural knowledge.

By Liang He, Jingbo Wen, Hongyu Gu, Hao Li, Haoyu Wang, Yixiong Chen, Kangning Cui, Xilu Wang

llms agents

arXiv AI

Jun 3

SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale

arXiv:2606. 03056v1 Announce Type: new Abstract: As LLM agents adopt large skill libraries, selecting the right subset becomes a structural problem rather than a similarity-matching one: skills depend on, conflict with, specialize, or duplicate one another, a structure invisible to both full enumeration and embedding similarity.

By Tong Bai, Zhenglin Wan, Pengfei Zhou, Xingrui Yu, Wangbo Zhao, Yang You, Ivor W. Tsang

llms rag agents diffusion

arXiv Computation and Language

Sep 30

Video2Skill: From Streaming Experience to Reusable Embodied Skills

arXiv:2609.36691v1 Announce Type: new Abstract: Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embod...

By Jianshu Zhang, Ce Zhang, Xiyuan Yang, Chenwei Xu, Haoran Lu, Yijiang Li, Yaqi Xie, Katia P. Sycara, Han Liu

llms agents robotics fine-tuning multimodal benchmarks

arXiv AI

Oct 1

SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale

arXiv:2609.38822v1 Announce Type: cross Abstract: Anthropic's Agent Skills package reusable procedural know-how for an LLM agent into SKILL.md directories, and open-source aggregations have grown pas...

By Guanqun Yang, Wenlong Zhang, Tian Shi, Ping Wang

llms agents benchmarks

arXiv Computation and Language

Aug 25

Signal or Noise? A Benchmark Study of Agent Skills in Web Development

arXiv:2608.23067v1 Announce Type: new Abstract: Agent Skills are reusable procedural modules that are increasingly injected into coding-agent sessions to encode framework conventions, anti-patterns,...

By Ziyue Yang, Fan Ding

agents benchmarks

Related stories