Cross-Layer Misalignment Detection in Agent Skills: A Progressive Loading-Aware Contrastive Learning Approach

2026-07-11

AI Summary

This work addresses the cross-layer misalignment between skill metadata and actual behaviors in large language model agents by proposing a Progressive Loading-aware Hierarchical Contrastive Learning framework (PL-HCL). It introduces hierarchical contrastive learning—applied for the first time to misalignment detection—to jointly model the hierarchical structure of skills and their dynamic loading process, thereby enabling cross-layer consistency verification. The method incorporates normalized processing of large-scale open-source skill corpora and achieves a substantial improvement in Macro-F1 score, raising it from the baseline of 0.45 to 0.87–0.89 on a dataset of 264,000 samples. Furthermore, the study distills key design principles for consistency validation tailored to hierarchical digital artifacts.

Abstract

Large language model (LLM) agents are increasingly extended through Agent Skills, reusable artifacts that package natural-language metadata, procedural instructions, and execution-time resources for runtime use. As open-source skill marketplaces expand, users and agents increasingly rely on brief metadata to select third-party skills, making it difficult to detect inconsistencies between a skill's description and its true behavior, a problem we call cross-layer misalignment. To address this issue, we propose Progressive Loading-Aware Hierarchical Contrastive Learning (PL-HCL), an LLM-based framework that detects misalignment by modeling the layered structure of Agent Skills and learning cross-layer consistency. Using a normalized corpus of over 264,000 open-source skills and a human-verified challenge set, PL-HCL improves Macro-F1 from approximately 0.45 for unadapted baselines to 0.87-0.89 across evaluated LLM backbones. This approach offers an effective screening tool for users and operators, as well as design principles for detecting inconsistencies in layered digital artifacts.

Problem

• cross-layer misalignment
• Agent Skills
• LLM agents
• metadata-behavior inconsistency
• skill marketplace

Innovation

• cross-layer misalignment
• progressive loading-aware contrastive learning
• agent skills
• hierarchical consistency
• LLM agents