Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms

2026-05-03

AI Summary

Existing adversarial imitation learning methods lack theoretical guarantees under general function approximation and suffer from algorithmic complexity that limits practical applicability. This work proposes OPT-AIL, an optimization-driven adversarial imitation learning framework that integrates online reward learning with optimistic regularized policy optimization, supporting both model-free and model-based instantiations. OPT-AIL is the first approach to achieve provably sample-efficient imitation under general function approximation, requiring only polynomial numbers of expert demonstrations and environment interactions. The method combines theoretical rigor with algorithmic simplicity. Empirical evaluations demonstrate that OPT-AIL outperforms current deep adversarial imitation learning algorithms across multiple challenging tasks while maintaining theoretically grounded sample and interaction efficiency.

Abstract

Adversarial imitation learning (AIL), a prominent approach in imitation learning, has achieved significant practical success powered by neural network approximation. However, existing theoretical analyses of AIL are primarily confined to simplified settings, such as tabular and linear function approximation, and involve complex algorithmic designs that impede practical implementation. This creates a substantial gap between theory and practice. This paper bridges this gap by exploring the theoretical underpinnings of online AIL with general function approximation. We introduce a novel framework called optimization-based AIL (OPT-AIL), which performs online optimization for reward learning coupled with optimism-regularized optimization for policy learning. Within this framework, we develop two concrete methods: model-free OPT-AIL and model-based OPT-AIL. Our theoretical analysis demonstrates that both variants achieve polynomial expert sample complexity and interaction complexity for learning near-expert policies. To the best of our knowledge, they represent the first provably efficient AIL methods under general function approximation. From a practical standpoint, OPT-AIL requires only the approximate optimization of two objectives, thereby facilitating practical implementation. Empirical studies demonstrate that OPT-AIL outperforms previous state-of-the-art deep AIL methods across several challenging tasks.

Problem

• Adversarial Imitation Learning
• General Function Approximation
• Theory-Practice Gap
• Sample Complexity
• Online Learning

Innovation

• Adversarial Imitation Learning
• General Function Approximation
• Optimization-based Framework
• Sample Complexity
• Optimism Regularization