MADR: MPC-guided Adversarial DeepReach

2025-10-21

AI Summary

Hamilton–Jacobi (HJ) reachability analysis suffers from the curse of dimensionality in high-dimensional adversarial settings, rendering it computationally intractable; meanwhile, physics-informed neural networks (PINNs) exhibit slow convergence and low accuracy in self-supervised training for differential games. Method: This paper proposes an MPC-guided deep learning framework that, for the first time, incorporates model predictive control (MPC)-generated supervision signals into deep reachability training for two-player zero-sum differential games. The method integrates PINNs, adversarial differential game theory, and a hybrid self-supervised/strongly-supervised training scheme, while enhancing PDE gradient guidance to improve learning dynamics. Contribution/Results: The proposed approach significantly improves both efficiency and accuracy in learning value functions and safety policies. Experiments demonstrate superior performance over state-of-the-art methods across multiple high-dimensional simulated and real-world robotic platforms, with rapid convergence, strong robustness, and real-time deployability.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Hamilton-Jacobi (HJ) Reachability offers a framework for generating safe value functions and policies in the face of adversarial disturbance, but is limited by the curse of dimensionality. Physics-informed deep learning is able to overcome this infeasibility, but itself suffers from slow and inaccurate convergence, primarily due to weak PDE gradients and the complexity of self-supervised learning. A few works, recently, have demonstrated that enriching the self-supervision process with regular supervision (based on the nature of the optimal control problem), greatly accelerates convergence and solution quality, however, these have been limited to single player problems and simple games. In this work, we introduce MADR: MPC-guided Adversarial DeepReach, a general framework to robustly approximate the two-player, zero-sum differential game value function. In doing so, MADR yields the corresponding optimal strategies for both players in zero-sum games as well as safe policies for worst-case robustness. We test MADR on a multitude of high-dimensional simulated and real robotic agents with varying dynamics and games, finding that our approach significantly out-performs state-of-the-art baselines in simulation and produces impressive results in hardware.

Problem

• Overcoming dimensionality curse in Hamilton-Jacobi reachability analysis
• Improving convergence of physics-informed deep learning for control
• Solving two-player zero-sum differential games with optimal strategies

Innovation

• Combines MPC guidance with adversarial deep learning
• Approximates two-player zero-sum differential game value
• Generates optimal strategies for worst-case robustness