Active Constraint Learning in High Dimensions from Demonstrations

2025-12-27

AI Summary

Inverse reinforcement learning for high-dimensional nonlinear systems under unknown environmental constraints remains challenging due to sparse, costly expert demonstrations and poor generalization. Method: We propose an iterative active constraint learning framework within the imitation learning paradigm. Leveraging Gaussian processes (GPs) to model unknown constraint functions, we introduce GP-based active learning—selecting informative start-end state queries via posterior uncertainty—to guide experts in generating information-rich demonstration trajectories. Our method integrates optimal control sampling with nonlinear dynamics simulation. Results: Evaluated on both simulation and real-robot platforms, our approach achieves high-fidelity constraint reconstruction and robust navigation generalization using only a few (sparse) iterative demonstrations—significantly outperforming random sampling baselines. It markedly improves data efficiency and modeling accuracy while maintaining scalability to high-dimensional, nonlinear systems.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

We present an iterative active constraint learning (ACL) algorithm, within the learning from demonstrations (LfD) paradigm, which intelligently solicits informative demonstration trajectories for inferring an unknown constraint in the demonstrator's environment. Our approach iteratively trains a Gaussian process (GP) on the available demonstration dataset to represent the unknown constraints, uses the resulting GP posterior to query start/goal states, and generates informative demonstrations which are added to the dataset. Across simulation and hardware experiments using high-dimensional nonlinear dynamics and unknown nonlinear constraints, our method outperforms a baseline, random-sampling based method at accurately performing constraint inference from an iteratively generated set of sparse but informative demonstrations.

Problem

• Active constraint learning from demonstrations in high dimensions
• Iterative algorithm for inferring unknown constraints using Gaussian processes
• Generates informative demonstrations to improve constraint inference accuracy

Innovation

• Iterative active constraint learning algorithm
• Gaussian process models unknown constraints
• Generates informative demonstrations for inference