Capability-Driven Skill Generation with LLMs: A RAG-Based Approach for Reusing Existing Libraries and Interfaces

2025-05-06

AI Summary

To address the inefficiency and time consumption inherent in skill development for autonomous systems, this paper proposes a capability-contract-driven LLM code generation framework. It formalizes capabilities as rigorous interface contracts (expressed in OWL/SWRL) to guide large language models in directly generating executable skill code from natural-language instructions. We introduce the first RAG framework tailored for skill engineering, enabling dynamic injection and semantic retrieval of user-defined libraries and heterogeneous interfaces (e.g., ROS 2). The framework further supports cross-language code synthesis and reuse between Python and C++. Evaluated on an autonomous mobile robot platform, the approach achieves an 82% one-shot success rate for navigation and perception skills, reducing average development time by 76%. These results demonstrate substantial improvements in both development efficiency and generalization capability of modular robotic skills.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Modern automation systems increasingly rely on modular architectures, with capabilities and skills as one solution approach. Capabilities define the functions of resources in a machine-readable form and skills provide the concrete implementations that realize those capabilities. However, the development of a skill implementation conforming to a corresponding capability remains a time-consuming and challenging task. In this paper, we present a method that treats capabilities as contracts for skill implementations and leverages large language models to generate executable code based on natural language user input. A key feature of our approach is the integration of existing software libraries and interface technologies, enabling the generation of skill implementations across different target languages. We introduce a framework that allows users to incorporate their own libraries and resource interfaces into the code generation process through a retrieval-augmented generation architecture. The proposed method is evaluated using an autonomous mobile robot controlled via Python and ROS 2, demonstrating the feasibility and flexibility of the approach.

Problem

• Generating executable code from natural language for skills
• Integrating existing libraries and interfaces across languages
• Reducing time-consuming skill development in modular systems

Innovation

• LLM-based code generation from natural language
• RAG architecture integrating existing libraries
• Multi-language skill implementation framework