Positive-Unlabeled Constraint Learning for Inferring Nonlinear Continuous Constraints Functions From Expert Demonstrations

2024-08-03

IEEE Robotics and Automation Letters

AI Summary

To address the challenge of enabling robots to complete tasks in unknown or difficult-to-model constrained environments, this paper proposes a Positive–Unlabeled Constraint Learning (PUCL) framework that automatically infers nonlinear continuous feasibility constraints from expert demonstrations—without requiring predefined constraint forms or environment dynamics models. The method follows a two-stage paradigm: it treats expert trajectories as positive samples and policy-sampled trajectories as unlabeled samples, integrating distance-guided reliable negative sample mining with binary feasibility classifier training. Notably, this work pioneers the application of PU learning to constraint discovery, enabling accurate modeling of complex non-convex constraint boundaries while preventing misclassification of expert behavior as infeasible. Evaluated across four diverse constrained environments, PUCL achieves significantly higher constraint recovery accuracy and policy safety compared to existing baselines.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Planning for diverse real-world robotic tasks necessitates to know and write all constraints. However, instances exist where these constraints are either unknown or challenging to specify accurately. A possible solution is to infer the unknown constraints from expert demonstration. This letter presents a novel two-step Positive-Unlabeled Constraint Learning (PUCL) algorithm to infer a continuous constraint function from demonstrations, without requiring prior knowledge of the true constraint parameterization or environmental model as existing works. We treat all data in demonstrations as positive (feasible) data, and learn a control policy to generate potentially infeasible trajectories, which serve as unlabeled data. The proposed two-step learning framework first identifies reliable infeasible data using a distance metric, and secondly learns a binary feasibility classifier (i.e., constraint function) from the feasible demonstrations and reliable infeasible data. The proposed method is flexible to learn complex-shaped constraint boundary and will not mistakenly classify demonstrations as infeasible as previous methods. The effectiveness of the proposed method is verified in four constrained environments, using a networked policy or a dynamical system policy. It successfully infers the continuous nonlinear constraints and outperforms other baseline methods in terms of constraint accuracy and policy safety.

Problem

• Robot Learning
• Complex Continuous Rules
• Unknown Situations

Innovation

• PUCL Algorithm
• Complex Continuous Rules Learning
• Expert Demonstration