An Empirical Study of Agent Skills' Downstream Utility

Key points

• Existing studies characterize Skill content and evaluate downstream performance, yet provide limited explanations of how utility depends on content, execution configuration, and multi-Skill organization.1
• We conduct an empirical study on 87 SkillsBench tasks, defining downstream utility as the pass-rate difference from No-Skill on the same tasks under the same model--harness configuration.1
• We compare the same Skills across nine configurations, then examine alternative published Skills and organizations of fixed Skill sets under three selected configurations.1
• We derive 17 authoring practices linking executable procedures to recovery, preservation of task requirements, and checks on final artifacts.1

Sources (1)

• [1]An Empirical Study of Agent Skills' Downstream Utility
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 06:05 AM

  Agent Skills package procedural guidance and resources for reuse, but a relevant Skill does not necessarily improve task performance.

  Existing studies characterize Skill content and evaluate downstream performance, yet provide limited explanations of how utility depends on content, execution configuration, and multi-Skill organization.

Extractive summary: sentences quoted from the sources.