SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing

2026-06-12

AI Summary

This work addresses the degradation of deployed agent skills due to edge cases, API changes, or environmental constraints by proposing an unsupervised skill evolution framework that operates without ground-truth labels or external rewards. The approach leverages a paired trajectory auditing mechanism to compare execution trajectories with and without a given skill, transforming behavioral discrepancies into actionable diagnostic signals through a structured validator and a Process-Aligned Contrastive Evaluation (PACE) module. These signals drive a dual-path editing strategy—Refine and Repair—to automatically update skill documentation. Evaluated across 89 containerized tasks spanning eight specialized domains, the method achieves an average task reward of 73.9%, substantially outperforming both non-skilled agents (40.9%) and static expert-provided skills (56.7%).

Abstract

Agent skills are structured procedural packages that guide frozen LLM agents in specialized workflows. Skills rarely remain sufficient after deployment: edge cases, API changes, and deployment constraints become visible only through use, making skill evolution a practical necessity. Existing methods depend on privileged feedback such as held-out validation scores, hidden test outcomes, or environment rewards -- signals often unavailable when a practitioner has only a task description and workspace data. We introduce SkillAudit, a framework for evolving agent skills without ground-truth feedback. The key idea is paired trajectory auditing: at each iteration, the same task is executed with and without the candidate skill, isolating how the skill changes agent behavior without external labels. To turn behavioral differences into edit guidance, SkillAudit uses Process-Aligned Contrastive Evaluation (PACE), a cluster of evaluators that maps trajectory divergences to diagnostic signals linked to specific passages in the skill document. A structural verifier, compiled once from the task specification and then fixed, checks task constraints and rolls back harmful updates. SkillAudit routes edits through two pipelines: Refine removes noisy or irrelevant guidance from broadly useful skills, while Repair replaces passages that conflict with the task. Across 89 containerized tasks spanning 8 professional domains, SkillAudit achieves 73.9% average task reward, outperforming an agent without skills (40.9%) and the static expert skill (56.7%). These gains are obtained without accessing hidden tests, reference solutions, or external scoring functions during evolution.

Problem

• skill evolution
• ground-truth-free
• agent skills
• trajectory auditing
• LLM agents

Innovation

• Skill Evolution
• Paired Trajectory Auditing
• Ground-Truth-Free Learning
• Process-Aligned Contrastive Evaluation
• Agent Skills