BooST: Bridging Semantics and Motions for Efficient Skill Transfer

2026-08-11

AI Summary

Existing approaches to robotic skill transfer often struggle to jointly capture semantic intent and motion dynamics, limiting their generalization, robustness, and deployment efficiency. This work proposes BooST, a two-stage framework that explicitly integrates semantics and dynamics for the first time. By leveraging a cross-modal Vector-Quantized Variational Autoencoder (VQ-VAE), BooST constructs a unified skill representation, which is then distilled—through skill abstraction and policy distillation—into a lightweight, deployable policy. Evaluated in both simulation and real-world robotic settings, the method demonstrates exceptional few-shot adaptation capabilities, strong cross-task and cross-domain transfer performance, and robustness to dynamic visual perturbations, all while maintaining computational efficiency for practical deployment.

Abstract

Skill abstraction---the process of learning reusable and temporally extended behaviors---has emerged as a key paradigm for improving sample efficiency and generalization in robot learning. For efficient skill transfer to real robots, learned skills must generalize across tasks and domains, remain robust to visual and dynamic perturbations, and be efficient enough for practical deployment. However, existing methods typically satisfy only a subset of these properties, as they capture either high-level semantic intent (what) or low-level motion dynamics (how). This incomplete skill transfer yields weak priors for policy learning, thereby demanding substantial in-domain data for downstream adaptation. To address these challenges, we introduce BooST, a two-stage framework that explicitly bridges semantics and motions to satisfy all three desiderata. BooST first leverages a cross-modal VQ-VAE to capture both semantic intent and motion dynamics, yielding a unified skill representation. It then distills this representation into a lightweight policy for efficient downstream adaptation to new tasks. Extensive experiments across simulation and real-robot settings demonstrate that BooST achieves superior few-shot adaptation, cross-domain skill transfer, and robustness to dynamic visual distractors, while maintaining a lightweight yet expressive design suitable for real-world deployment.

Problem

• skill transfer
• semantic intent
• motion dynamics
• sample efficiency
• cross-domain generalization

Innovation

• skill abstraction
• cross-modal VQ-VAE
• semantic-motion bridging
• few-shot adaptation
• efficient skill transfer