SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces

2026-07-02

AI Summary

This work addresses the risk that seemingly benign skill compositions in open skill markets can implicitly generate intentions misaligned with user goals—risks often undetectable by existing auditing mechanisms. The paper introduces the first execution-free fuzzing approach for skill composition: leveraging structured skill contracts and a differential oracle, it statically detects compositional risks by comparing planning outputs against a no-skill baseline. Furthermore, it devises a contract-guided Monte Carlo Tree Search to efficiently explore high-risk regions of the combinatorial space. Under a fixed query budget, experiments demonstrate that the method uncovers over 1,000 implicit intents, with execution-based validation confirming more than 80% as high-risk compositions—significantly outperforming baseline strategies while exploring only a minuscule fraction of the total composition space.

Abstract

Large Language Model (LLM)-based agents increasingly automate software engineering tasks through reusable skills, natural-language instruction documents that guide planning and execution. Open skill marketplaces enable users to assemble agents by co-activating community-contributed skills, but marketplace operators typically audit skills in isolation. As a result, individually benign skills may interact to redirect an agent toward unintended objectives, which we term implicit intents. Detecting such intents is challenging because the effect emerges only through skill composition, execution environments are often unavailable at admission time, and the space of possible co-activations grows exponentially with marketplace size. In this paper, we formulate implicit-intent discovery as a fuzzing problem over skill compositions, where skill compositions are the unit under test, planning artifacts expose agent intent before execution, and deviations from a skill-free baseline serve as a differential oracle. Based on this formulation, we propose skillfuzz, the first execution-free testing approach that extracts structured skill contracts and uses contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions. Across representative skill-marketplace workloads, skillfuzz discovers over 1,000 distinct implicit intents under a fixed query budget, confirms more than 80% of the highest-risk flagged compositions during execution-time validation, and identifies substantially more high-severity implicit intents than alternative search strategies while exploring only a fraction of the pairwise interaction space they require.

Problem

• implicit intents
• skill composition
• open skill marketplaces
• fuzzing
• LLM-based agents

Innovation

• implicit intents
• skill composition
• fuzzing
• execution-free testing
• Monte Carlo Tree Search