arXiv AI
What Does a Skill Actually Do? Estimands and Evaluation Validity for Tool and Skill Use in LLM Agents: A Critical Review
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
3d ago
What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents
The article reviews how replacing components in language‑model agents affects execution trajectories and downstream outcomes. It maps 348 studies, analyzes 90 comparison records, and finds that while many studies report both local decision metrics and task endpoints, they rarely demonstrate matched comparisons or prove that improved local decisions drive task‑level gains. The review identifies three potential mechanisms—recovery and disruption, intervention timing, and downstream use—and proposes eight claim‑specific reporting items to clarify evidence quality.
arXiv AI
Jun 11
SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior
arXiv:2606. 11543v1 Announce Type: new Abstract: Agent Skills augment large language model (LLM) agents with procedural knowledge at inference time, but current benchmarks rarely distinguish what a Skill says from how it is organized.
Hugging Face Trending Papers
Jun 10
SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior
Agent Skills augment large language model (LLM) agents with procedural knowledge at inference time, but current benchmarks rarely distinguish what a Skill says from how it is organized. We study this distinction through Progressive Disclosure, where a concise root file points agents to supporting resources on demand, and compare it with a normalized flat baseline.
arXiv AI
Aug 13
Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents
arXiv:2608. 11888v1 Announce Type: new Abstract: Agent skills are the de facto mechanism for extending LLM agents with reusable guidance.
arXiv AI
Jun 16
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
arXiv:2602. 12670v4 Announce Type: replace Abstract: Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time.
Hugging Face Trending Papers
Aug 12
Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents
Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving, and validation.