Ro-SLM: Onboard Small Language Models for Robot Task Planning and Operation Code Generation

2026-04-12

AI Summary

This work addresses the challenge of deploying large language models (LLMs) on resource-constrained and network-unstable robotic platforms, such as drones, where reliance on cloud infrastructure or high-performance computing is impractical. The authors propose Ro-SLM, a novel framework that transfers LLM capabilities to lightweight models through knowledge distillation. Ro-SLM innovatively leverages the LLM to generate synthetic training data and serve as a reward function to guide fine-tuning. By integrating data synthesis, instruction augmentation, supervised fine-tuning, and LLM-guided reinforcement learning, the approach substantially enhances the small model’s performance in robotic task planning and code generation—elevating it from near-complete failure to near-parity with full-scale LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent advances in large language models (LLMs) provide robots with contextual reasoning abilities to comprehend human instructions. Yet, current LLM-enabled robots typically depend on cloud-based models or high-performance computing infrastructure, which limit their deployment on robots under unreliable internet environments or with constrained computational resources, such as UAVs and small ground vehicles. Thus, deploying fine-tuned small language models (SLMs) that support onboard deployment offers a promising alternative. This paper introduces Ro-SLM, a framework that enables reliable SLM-driven robot operation by distilling LLMs' knowledge and reasoning. Ro-SLM starts from dataset synthesis by leveraging LLMs to generate diverse task instructions, produce corresponding ground truth code with minimal human assistance, and augment instructions into real-world application scenarios. Ro-SLM is then fine-tuned with the dataset, in which LLM serves as a reward function to guide the training. Extensive experiments on UAV operation tasks demonstrate that Ro-SLM improves the performance of SLM from being incapable of supporting robotic task planning and code generation to achieving performance that approaches LLM.

Problem

• onboard deployment
• small language models
• robot task planning
• code generation
• resource-constrained robots

Innovation

• Small Language Models
• Onboard Deployment
• Knowledge Distillation
• Robot Task Planning
• Code Generation