OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models

2026-06-15

AI Summary

This work proposes the Collective Skill Tree Search (CSTS) framework to address the challenges of tool use, multi-step reasoning, and dynamic interaction faced by large language model agents in complex tasks. CSTS leverages collective intelligence to iteratively generate and evaluate skill nodes, constructing a structured and reusable skill tree. It integrates skill-guided reinforcement learning for adaptive skill selection and introduces a novel dual-scoring mechanism that assesses both collective quality and transferability. By combining multi-model collaborative reasoning, CSTS significantly enhances planning capabilities and cross-task generalization. Experimental results demonstrate that OpenClaw-Skill, trained with CSTS, achieves state-of-the-art performance on benchmarks involving long-horizon planning and tool invocation, effectively mitigating suboptimal solutions caused by reliance on individual skills.

Abstract

Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems like OpenClaw. In this work, we aim to develop a framework that automatically constructs such reusable skills to enhance LLMs in tool use, multi-step reasoning, and dynamic environment interaction. To this end, we propose Collective Skill Tree Search (CSTS), a novel tree-search-based skill construction framework that constructs structured, diverse and generalizable tree of skills. The core idea of CSTS is to leverage collective intelligence to jointly search, identify and compose effective skills via two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess). CSN-Gen exploits collective knowledge from multiple models to explore diverse candidate skills for each subtask, enabling comprehensive skill exploration. CSN-Assess employs multiple models as judges to evaluate and select skill nodes with two scoring mechanisms: (1) collective quality scoring that aggregates independent evaluations to produce a robust estimate of skill effectiveness, and (2) collective transferability scoring that explicitly verifies whether a skill generalizes well across different models. With CSTS, we construct a set of comprehensive tree of skills along with skill-augmented training data, enabling models to effectively learn and utilize skills. Besides, we introduce Collective Skill Reinforcement Learning, which actively selects multiple relevant skills from the tree to broaden solution-space exploration, avoid being trapped by a single skill and its resulting homogeneous or suboptimal solutions. As a result, our trained model, OpenClaw-Skill, exhibits outstanding agentic capabilities in long-horizon planning, tool use and generalization over challenging benchmarks.

Problem

• skill acquisition
• agentic LLMs
• tool use
• multi-step reasoning
• generalization

Innovation

• Collective Skill Tree Search
• Skill Construction
• Agentic LLMs
• Collective Intelligence
• Skill Transferability