Large Language Models to Enhance Multi-task Drone Operations in Simulated Environments
This work proposes a natural language–based control framework to lower the barrier to multi-task drone operation. By fine-tuning the CodeT5 model on (natural language instruction, executable code) pairs generated by ChatGPT, the system automatically translates user commands into executable scripts that drive drones in a high-fidelity AirSim/Unreal Engine simulation environment. This study presents the first integration of large language models with a high-fidelity drone simulation platform, enabling complex task execution through intuitive linguistic input. Experimental results demonstrate that the approach achieves strong instruction comprehension and reliable task execution in simulation, significantly enhancing human–drone interaction efficiency and laying a foundation for real-world natural language control of autonomous aerial systems.