Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces AIDE, a novel framework for generating natural-language coaching feedback on motor skills that relies solely on a learner's pose sequence during inference, eliminating the need for expert demonstrations at that stage. By training a teacher model to generate feedback from paired learner-expert poses and then distilling this knowledge into a student model, AIDE effectively captures the necessary feedback without direct reference to expert input during deployment. The results on the ExpertAF dataset show that AIDE not only surpasses existing reference-free methods but also matches the performance of those requiring expert demonstrations, highlighting its practical applicability in coaching scenarios.
AIDE achieves expert-level motor skill coaching feedback without needing real-time access to expert demonstrations, revolutionizing how we approach skill acquisition.
Generating natural-language coaching feedback on motor skills can accelerate learning, yet expert coaches are scarce and expensive. Existing reference-based methods require expert demonstrations at both training and inference time, limiting practical deployment. We propose AIDE (Automated Instruction via Distilled Expertise), a framework that exploits expert references only during training and generates feedback from a learner's pose sequence alone at inference. A teacher model first learns to generate feedback from paired learner-expert poses via a frozen language model, producing separate learner tokens and difference tokens that encode the learner-expert difference. A student model then inherits the teacher's encoder and weight initialization, replacing the explicit expert comparison with an auxiliary module that produces complementary tokens from the learner's pose alone. On the ExpertAF dataset, AIDE outperforms reference-free baselines on most metrics and performs comparably to methods requiring expert demonstrations at both training and inference, with LLM-based evaluation supporting these findings.