What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files

2026-08-08

AI Summary

This study addresses the gap between the claimed reusability and actual cross-scenario transferability of publicly available agent skills (SKILL.md), which often originate from single-task contexts. Building upon official specifications, we establish a two-tier defect taxonomy and conduct the first large-scale empirical analysis of 138,133 skill files, revealing that reuse barriers stem predominantly from routine packaging flaws rather than sophisticated attacks. We propose a quality-assured generation pipeline integrating specification-aware prompting, lightweight linting, automated repair, and security gating. Our experiments show that 91.8% of skills contain at least one defect; skills with valid routing metadata achieve significantly higher retrieval success rates; specification-aware skills exhibit fewer defects overall, whereas AI-generated skills suffer from more severe security and portability issues.

Abstract

Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single conversation. Yet many public skills appear to originate from a single task, repository, or conversation, even when they are shared as reusable components. We analyze this gap across 138,133 public SKILL.md files from 20,556 repositories using a two-tier defect taxonomy grounded in the official specification and best-practice guidance. We find that 91.8% of skills contain at least one detected defect, with stable estimates across lenient and strict thresholds (88.8-94.6%). The dominant failures are ordinary packaging problems rather than exotic attacks: weak routing metadata, bloated or non-actionable bodies, and poor resource organization. A deterministic routing stress test over 20,000 skills shows the functional impact: skills with valid routing metadata are retrieved more reliably from startup descriptions than skills with routing defects. Defect rates vary by platform and provenance: specification-aware skills contain fewer defects, while AI-marked skills show more safety and portability problems. Lightweight enforcement and repair experiments support a quality-assured generation workflow combining spec-aware prompting, lightweight linting, automated repair, and safety gating.

Problem

• Agent Skills
• reusability
• SKILL.md
• defect taxonomy
• routing metadata

Innovation

• Agent Skills
• SKILL.md
• defect taxonomy
• routing metadata
• quality-assured generation