SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution

2026-05-11

AI Summary

Existing Retrieval-Augmented Execution (RAE) approaches struggle to effectively organize retrieved skill evidence when scaling skill libraries, often producing redundant and unreliable contextual information. To address this limitation, this work proposes SkillRAE, a two-stage RAE framework that first constructs a multi-level skill graph offline and then, at inference time, integrates skill-aware retrieval with sub-unit evidence extraction. SkillRAE further introduces a rescue-aware compact compilation strategy to generate high-quality, task-oriented execution contexts. As the first method specifically designed for skill-oriented context compilation, SkillRAE substantially outperforms existing approaches on benchmarks such as SkillsBench, achieving an 11.7% performance gain. Ablation studies confirm the critical contribution of the proposed context compilation module to overall effectiveness.

Abstract

Large Language Model (LLM)-based agents (e.g., OpenClaw) increasingly rely on reusable skill libraries to solve artifact-rich tasks such as document-centric workflows and data-intensive analysis. As these libraries grow, a few works have attempted to study the Retrieval-Augmented Execution (RAE), which often first retrieves some external skills and other knowledge, then compiles the context using retrieved skills, and finally executes the task. Existing works mainly focus on optimizing skill retrieval and task execution, and they pay little attention to how to effectively organize the selected skill evidence in a form that is compact, grounded, and immediately usable for the downstream executors to complete tasks. To fill this gap, we propose SkillRAE, a two-stage RAE approach focusing on skill-based context compilation, which consists of the offline and online stages. Specifically, in the offline indexing stage, it builds a multi-level skill graph over skill communities, skills, and reusable subunits, for capturing their relationships. In the online retrieval stage, it first performs skill-ranked retrieval with selected-subunit evidence export in the graph, and then applies rescue-aware compact compilation to recover the key evidence. Together, these components compile a coarse-ranked skill set into a task-specific context that is compact, grounded, and immediately usable. Experiments on two public benchmarks show that SkillRAE achieves a significant improvement over baselines for RAE. For example, on SkillsBench, it achieves an improvement of 11.7% over the SOTA method. Ablation studies further show that our context compilation is crucial, instead of a mere prompt addition.

Problem

• Retrieval-Augmented Execution
• skill library
• context compilation
• LLM-based agents
• task execution

Innovation

• Retrieval-Augmented Execution
• Skill-Based Context Compilation
• Multi-Level Skill Graph
• Rescue-Aware Compilation
• LLM Agents