From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills

2026-07-01

AI Summary

This study addresses the lack of systematic understanding regarding how SKILL.md files—documentation artifacts specifying agent capabilities—are actually authored in practice and how they align with recommended guidelines. Treating SKILL.md as a software artifact, the authors conduct a qualitative content analysis of 238 real-world instances, informed by a multi-source literature review, to develop a semantic component taxonomy, distill best practices for authoring, and introduce the novel concept of “skill smells” to denote common violations of documentation norms. They further devise an automated detection method, revealing that over 99% of SKILL.md files exhibit at least one skill smell. Crucially, these defects are rarely remedied during skill evolution, exposing a substantial gap between actual development practices and prescribed standards and underscoring the urgent need for improved quality governance.

Abstract

Agent Skills provide on-demand domain knowledge to LLM agents without requiring model retraining. Each Agent Skill is defined by a mandatory SKILL.md file containing metadata and an unstructured Markdown body whose contents are left entirely to the skill author. Despite the rapid adoption of Agent Skills, little is known about how these files are authored or whether existing authoring guidelines are followed in practice. In this paper, we present the first systematic study of SKILL.md files as a software artifact. We qualitatively analyze 238 real-world skills and derive a taxonomy of 13 higher-level and 44 lower-level semantic components. We then conduct a multivocal literature review of 29 sources to identify best practices for authoring SKILL.md files and introduce skill smells as violations of these practices. Finally, we develop an automated detector and apply it to real-world skills, finding that over 99% of SKILL.md files contain at least one skill smell, and once introduced, skill smells rarely disappear as skills evolve. These findings reveal a substantial gap between recommended and actual authoring practices, motivating the development of automated techniques to remediate skill smells while increasing developer awareness of this emerging quality issue.

Problem

• Agent Skills
• SKILL.md
• skill smells
• authoring practices
• software artifact

Innovation

• Agent Skills
• SKILL.md
• skill smells
• empirical study
• automated detection