To Select or not to Select, that is the Question: Distilling Robot Skill Prediction into a Small Ensemble

2026-05-20

AI Summary

This work addresses the challenge of robot selection in heterogeneous robotic swarms based on natural language task descriptions by proposing a lightweight, domain-specific modeling approach. The authors construct the first LLM-assisted synthetic dataset aligning tasks with required skills, followed by targeted label curation and sentence embedding fine-tuning to train an ensemble of two 133M-parameter models based on MPNet and MiniLM. Evaluated on a fixed skill set across a hierarchical test suite of 200 tasks, the proposed method achieves a skill-matching accuracy of 83.5%, significantly outperforming zero-shot baselines including Kimi K2 (72.0%), GPT-OSS-120B (71.5%), and Llama-4-Scout-17B (69.0%). These results demonstrate that compact, specialized models can surpass the zero-shot performance of much larger general-purpose language models in this domain.

Abstract

As robot fleets become more heterogeneous, including humanoids, rovers, quadrupeds, and drones, selecting the right robot for a task becomes a core systems problem. We study robot skill prediction: mapping a natural-language task description to the physical capabilities required to execute it, such as fly, wheels, legs, surface water, under water and hands. Since labelled data that maps natural-language task descriptions to robot's physical capabilities does not exist, we construct a synthetic task-to-skill dataset using LLM-assisted generation and targeted label auditing. Trained on this data, a ~133M-parameter ensemble of two fine-tuned sentence encoders (mpnet + MiniLM) reaches 83.5% task-to-skill matching on a stratified 200 task dataset, outperforming Kimi K2 (1T MoE) at 72.0%, GPT-OSS-120B at 71.5%, and Llama-4-Scout-17B at 69.0% under the same zero-shot prompt. These results suggest that, for fixed robot skill taxonomies, small specialized models trained on synthetic data can outperform much larger general-purpose LLMs for fleet-level task routing.

Problem

• robot skill prediction
• task-to-skill mapping
• heterogeneous robot fleets
• natural-language task description
• robot selection

Innovation

• robot skill prediction
• synthetic data generation
• small ensemble models
• LLM-assisted labeling
• task-to-skill mapping