SkillShapley: Boundary-Adaptive Shapley Valuation for Skill Step Attribution in LLM Agents

2026-08-13

AI Summary

Existing approaches struggle to quantify the contribution of individual steps within large language model agent skills to overall performance. To address this, this work proposes SkillShapley, a framework that formulates step attribution as a Shapley value–based contribution estimation problem. It introduces a novel boundary-adaptive Shapley valuation mechanism that leverages the performance cliff phenomenon and the approximately additive interactions among steps. The method operates in two stages: first identifying information-rich coalition regions, then adaptively sampling coalitions that yield reusable marginal evidence, thereby significantly enhancing attribution efficiency and interpretability. Experiments on the SkillsBench benchmark demonstrate that SkillShapley effectively identifies high- and low-value skill steps, offering practical guidance for agent skill design.

Abstract

Agent skills are crucial external instructions that enable language agents to execute long procedural tasks such as coding or document processing. Existing agent skills are primarily created through human manual crafting or agent execution traces, with limited understanding of how each step contributes to overall skill performance on specific tasks; i.e., there remains an open problem in quantifying the contribution of individual steps within an agent skill. To address this issue, we first model skill-step attribution as a Shapley value-based contribution estimation problem, and then propose SkillShapley, a step-level attribution framework for agent skills. Notably, SkillShapley operates in two phases, motivated by key empirical insights, i.e., discretized benchmark rewards that create sharp performance cliffs, and step interactions that are largely additive rather than synergistic. Specifically, it first identifies informative coalitional regions, and then adaptively samples new coalitions that can yield reusable marginal evidence. Experiments on skills from the widely adopted SkillsBench demonstrate that our SkillShapley can effectively and efficiently identify high- or low-value skill steps, providing several key takeaways for agent skill creation.

Problem

• skill step attribution
• Shapley value
• LLM agents
• contribution quantification
• agent skills

Innovation

• Shapley value
• skill attribution
• boundary-adaptive sampling
• agent skills
• step-level contribution