Learning Parameterized Skills from Demonstrations

2025-10-28

AI Summary

This work addresses zero-shot transfer to unseen tasks by learning generalized, parameterized skills from multi-task expert demonstrations. We propose an end-to-end hierarchical framework that jointly learns (i) a low-level parameterized skill policy and (ii) a high-level meta-policy for discrete skill selection and continuous parameter generation. Temporal dynamics of skills are modeled via temporal variational inference, while an information-theoretic regularization mitigates latent variable collapse—ensuring semantically distinct, temporally extended, and interpretable skills. The approach integrates deep reinforcement learning, variational inference, and hierarchical policy modeling. On the LIBERO and MetaWorld benchmarks, our method significantly outperforms both multi-task learning and state-of-the-art skill learning baselines. It discovers physically meaningful, parameterized skills—e.g., “grasping at specified locations”—demonstrating strong generalization and intrinsic interpretability in a unified framework.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present DEPS, an end-to-end algorithm for discovering parameterized skills from expert demonstrations. Our method learns parameterized skill policies jointly with a meta-policy that selects the appropriate discrete skill and continuous parameters at each timestep. Using a combination of temporal variational inference and information-theoretic regularization methods, we address the challenge of degeneracy common in latent variable models, ensuring that the learned skills are temporally extended, semantically meaningful, and adaptable. We empirically show that learning parameterized skills from multitask expert demonstrations significantly improves generalization to unseen tasks. Our method outperforms multitask as well as skill learning baselines on both LIBERO and MetaWorld benchmarks. We also demonstrate that DEPS discovers interpretable parameterized skills, such as an object grasping skill whose continuous arguments define the grasp location.

Problem

• Learning parameterized skills from multitask expert demonstrations
• Addressing degeneracy in latent variable skill models
• Improving generalization to unseen tasks through skill discovery

Innovation

• Learning parameterized skills from expert demonstrations
• Using variational inference and information-theoretic regularization
• Jointly learning skill policies with meta-policy selection