Hybrid Distillation with CoT Guidance for Edge-Drone Control Code Generation

2026-01-13

AI Summary

This work addresses the tension between the high computational demands of large language models and the stringent real-time requirements of resource-constrained drone platforms by proposing a lightweight approach that integrates knowledge distillation, chain-of-thought guidance, and supervised fine-tuning. A high-quality dataset comprising instruction-code-reasoning chains and counterfactual negative samples is constructed, and a hybrid distillation framework combining black-box and white-box strategies is designed. Furthermore, a prompt-tuning mechanism tailored for multi-SDK drone control is introduced. Using QLoRA-quantized DeepSeek-Coder-V2-Lite as the teacher model, distillation is performed with chain-of-thought soft labels and weighted cross-entropy loss. Experiments demonstrate that the resulting compact model maintains high code generation accuracy while significantly improving inference speed and deployment efficiency, thereby validating the effectiveness and superiority of the proposed method in intelligent drone control.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

With large language models demonstrating significant potential in code generation tasks, their application to onboard control of resource-constrained Unmanned Aerial Vehicles has emerged as an important research direction. However, a notable contradiction exists between the high resource consumption of large models and the real-time, lightweight requirements of UAV platforms. This paper proposes an integrated approach that combines knowledge distillation, chain-of-thought guidance, and supervised fine-tuning for UAV multi-SDK control tasks, aiming to efficiently transfer complex reasoning and code generation capabilities to smaller models. Firstly, a high-quality dataset covering various mainstream UAV SDKs is constructed, featuring instruction-code-reasoning chains, and incorporates counterfactual negative samples for data augmentation, guiding the model to learn the end-to-end logic from instruction parsing to code generation. Secondly, leveraging DeepSeek-Coder-V2-Lite quantized via QLoRA as the teacher model, and based on a hybrid black-box and white-box distillation strategy, high-quality chain-of-thought soft labels are generated. These are combined with a weighted cross-entropy loss using hard labels to transfer complex reasoning capabilities to the smaller student model. Finally, through prompt tuning engineering optimized for the UAV control scenario, the model performance on core tasks such as SDK type recognition and function call matching is enhanced. Experimental results indicate that the distilled lightweight model maintains high code generation accuracy while achieving significant improvements in deployment and inference efficiency, effectively demonstrating the feasibility and superiority of our approach in achieving precise and lightweight intelligent control for UAVs

Problem

• UAV control
• code generation
• resource-constrained
• large language models
• edge computing

Innovation

• Knowledge Distillation
• Chain-of-Thought
• Edge AI
• UAV Control
• Code Generation