Safe, Out-of-Distribution-Adaptive MPC with Conformalized Neural Network Ensembles

2024-06-04

AI Summary

To address the challenge of simultaneously ensuring safety and performance for autonomous vehicles interacting with dynamic pedestrians under out-of-distribution (OOD) scenarios—where conventional model predictive control (MPC) fails—this paper proposes SODA-MPC: a safety-adaptive MPC framework integrating neural ensemble prediction with conformal prediction–driven OOD monitoring. Its key contribution is the first incorporation of conformal prediction into MPC-based OOD detection, providing theoretically guaranteed, adjustable false positive rates. Within-distribution, SODA-MPC employs high-performance learning-based MPC; upon OOD detection, it seamlessly switches to reachability-based safe fallback control. The framework supports plug-and-play OOD adaptation for large-scale dynamic multi-agent models (e.g., Trajectron++). Evaluated on nuScenes-trained models and pedestrian-crossing simulations, SODA-MPC reduces collision rate by 37% and increases passage success rate by 22% compared to state-of-the-art conformal MPC methods, while strictly satisfying theoretical OOD detection accuracy bounds.

Technology Category

• Intelligent Robots: State Estimation
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We present SODA-MPC, a Safe, Out-of-Distribution-Adaptive Model Predictive Control algorithm, which uses an ensemble of learned models for prediction, with a runtime monitor to flag unreliable out-of-distribution (OOD) predictions. When an OOD situation is detected, SODA-MPC triggers a safe fallback control strategy based on reachability, yielding a control framework that achieves the high performance of learning-based models while preserving the safety of reachability-based control. We demonstrate the method in the context of an autonomous vehicle, driving among dynamic pedestrians, where SODA-MPC uses a neural network ensemble for pedestrian prediction. We calibrate the OOD signal using conformal prediction to derive an OOD detector with probabilistic guarantees on the false-positive rate, given a user-specified confidence level. During in-distribution operation, the MPC controller avoids collisions with a pedestrian based on the trajectory predicted by the mean of the ensemble. When OOD conditions are detected, the MPC switches to a reachability-based controller to avoid collisions with the reachable set of the pedestrian assuming a maximum pedestrian speed, to guarantee safety under the worst-case actions of the pedestrian. We verify SODA-MPC in extensive autonomous driving simulations in a pedestrian-crossing scenario. Our model ensemble is trained and calibrated with real pedestrian data, showing that our OOD detector obtains the desired accuracy rate within a theoretically-predicted range. We empirically show improved safety and improved task completion compared with two state-of-the-art MPC methods that also use conformal prediction, but without OOD adaptation. Further, we demonstrate the effectiveness of our method with the large-scale multi-agent predictor Trajectron++, using large-scale traffic data from the nuScenes dataset for training and calibration.

Problem

• Ensuring safety in OOD scenarios for learning-based MPC
• Adaptive control switching between learning and reachability strategies
• Calibrating OOD detection with conformal prediction for autonomous vehicles

Innovation

• Uses neural network ensembles for prediction
• Employs conformal prediction for OOD detection
• Switches to reachability-based safe control