Offline Discovery of Interpretable Skills from Multi-Task Trajectories

2026-02-01

AI Summary

This work proposes LOKI, a framework for reusable and interpretable skill discovery and hierarchical imitation from long-horizon, multi-task offline trajectories lacking explicit rewards or subtask annotations. LOKI employs a three-stage end-to-end learning pipeline: it first uses weak task labels to segment trajectories at a macro level, then integrates an alignment-augmented vector-quantized VAE with self-supervised sequence modeling to jointly refine skill boundaries through iterative clustering, and finally constructs an option-based hierarchical policy with learnable termination conditions. LOKI is the first method to achieve semantically interpretable and composable skill discovery in the reward-free, multi-task offline setting. On the D4RL Kitchen benchmark, it outperforms existing hierarchical imitation learning approaches, yielding skills that align with human intuition and can be effectively composed to solve novel tasks.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Hierarchical Imitation Learning is a powerful paradigm for acquiring complex robot behaviors from demonstrations. A central challenge, however, lies in discovering reusable skills from long-horizon, multi-task offline data, especially when the data lacks explicit rewards or subtask annotations. In this work, we introduce LOKI, a three-stage end-to-end learning framework designed for offline skill discovery and hierarchical imitation. The framework commences with a two-stage, weakly supervised skill discovery process: Stage one performs coarse, task-aware macro-segmentation by employing an alignment-enforced Vector Quantized VAE guided by weak task labels. Stage two then refines these segments at a micro-level using a self-supervised sequential model, followed by an iterative clustering process to consolidate skill boundaries. The third stage then leverages these precise boundaries to construct a hierarchical policy within an option-based framework-complete with a learned termination condition beta for explicit skill switching. LOKI achieves high success rates on the challenging D4RL Kitchen benchmark and outperforms standard HIL baselines. Furthermore, we demonstrate that the discovered skills are semantically meaningful, aligning with human intuition, and exhibit compositionality by successfully sequencing them to solve a novel, unseen task.

Problem

• offline skill discovery
• hierarchical imitation learning
• multi-task trajectories
• interpretable skills
• long-horizon tasks

Innovation

• Offline Skill Discovery
• Hierarchical Imitation Learning
• Vector Quantized VAE
• Option Framework
• Self-supervised Segmentation