AIDE: Automated Instruction via Distilled Expertise for Reference-Free Motor Skill Coaching

2026-08-03

AI Summary

Existing motion skill coaching methods rely on real-time expert demonstrations, resulting in high deployment costs and limited scalability. This work proposes AIDE, a novel framework that, for the first time, leverages expert demonstrations only during training via knowledge distillation and, at inference time, generates high-quality natural language feedback using solely the learner’s pose sequence—eliminating any dependence on real-time expert data. AIDE employs a teacher–student architecture: the teacher model, built upon a frozen large language model, extracts discrepancy encodings from paired learner–expert poses; the student model inherits this encoder and incorporates an auxiliary module to predict complementary information from a single-stream input. Evaluated on the ExpertAF dataset, AIDE significantly outperforms reference-free baselines and matches the performance of two-stage methods requiring expert demonstrations, with further validation provided by LLM-based assessments.

Abstract

Generating natural-language coaching feedback on motor skills can accelerate learning, yet expert coaches are scarce and expensive. Existing reference-based methods require expert demonstrations at both training and inference time, limiting practical deployment. We propose AIDE (Automated Instruction via Distilled Expertise), a framework that exploits expert references only during training and generates feedback from a learner's pose sequence alone at inference. A teacher model first learns to generate feedback from paired learner-expert poses via a frozen language model, producing separate learner tokens and difference tokens that encode the learner-expert difference. A student model then inherits the teacher's encoder and weight initialization, replacing the explicit expert comparison with an auxiliary module that produces complementary tokens from the learner's pose alone. On the ExpertAF dataset, AIDE outperforms reference-free baselines on most metrics and performs comparably to methods requiring expert demonstrations at both training and inference, with LLM-based evaluation supporting these findings.

Problem

• motor skill coaching
• reference-free feedback
• automated instruction
• expert demonstration
• pose-based feedback

Innovation

• reference-free coaching
• distilled expertise
• pose-based feedback generation
• teacher-student framework
• motor skill learning