Interpretable Generative Adversarial Imitation Learning

2024-02-15

arXiv.org

AI Summary

To address the challenge of opaque task semantics in imitation learning, this paper proposes an interpretable imitation learning framework integrating Signal Temporal Logic (STL) inference and control synthesis. Methodologically: (1) expert demonstrations are explicitly encoded as human-readable and tunable STL formulas; (2) an STL-guided feedback controller is designed; and (3) a generative adversarial joint training scheme is introduced to end-to-end co-optimize the STL inference module and the controller. The key contribution is the first end-to-end interpretable modeling of task logic in imitation learning—enabling incorporation of human prior knowledge and manual formula refinement, thus balancing automation with human controllability. In two case studies, the method significantly reduces behavioral divergence between expert demonstrations and learned policies, empirically validating its interpretability, generalization capability, and adaptability across diverse scenarios.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Imitation learning methods have demonstrated considerable success in teaching autonomous systems complex tasks through expert demonstrations. However, a limitation of these methods is their lack of interpretability, particularly in understanding the specific task the learning agent aims to accomplish. In this paper, we propose a novel imitation learning method that combines Signal Temporal Logic (STL) inference and control synthesis, enabling the explicit representation of the task as an STL formula. This approach not only provides a clear understanding of the task but also allows for the incorporation of human knowledge and adaptation to new scenarios through manual adjustments of the STL formulae. Additionally, we employ a Generative Adversarial Network (GAN)-inspired training approach for both the inference and the control policy, effectively narrowing the gap between the expert and learned policies. The effectiveness of our algorithm is demonstrated through two case studies, showcasing its practical applicability and adaptability.

Problem

• Lack of interpretability in imitation learning methods
• Need for explicit task representation using STL formulas
• Adaptation to out-of-distribution scenarios via manual adjustments

Innovation

• Combines STL inference and control synthesis
• Uses GAN-inspired training for policy networks
• Enables manual STL adjustment for adaptation