Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models

2026-02-12

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Reinforcement learning (RL) is a powerful framework for optimal decision-making and control but often lacks provable guarantees for safety-critical applications. In this paper, we introduce a novel recovery-based shielding framework that enables safe RL with a provable safety lower bound for unknown and non-linear continuous dynamical systems. The proposed approach integrates a backup policy (shield) with the RL agent, leveraging Gaussian process (GP) based uncertainty quantification to predict potential violations of safety constraints, dynamically recovering to safe trajectories only when necessary. Experience gathered by the'shielded'agent is used to construct the GP models, with policy optimization via internal model-based sampling - enabling unrestricted exploration and sample efficient learning, without compromising safety. Empirically our approach demonstrates strong performance and strict safety-compliance on a suite of continuous control environments.

Problem

• Safe Reinforcement Learning
• Safety Guarantees
• Continuous Dynamical Systems
• Safety Constraints
• Recovery-based Shielding

Innovation

• Recovery-based Shielding
• Gaussian Process Dynamics
• Safe Reinforcement Learning
• Uncertainty Quantification
• Model-based Sampling