How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints

2025-11-23

AI Summary

Existing safety filters for high-dimensional perceptual inputs suffer from non-smooth control due to difficulties in modeling constraints—current approaches rely on discrete switching, degrading task performance. Method: We propose LatentCBF, a framework that jointly learns a Control Barrier Function (CBF) and a value function in a latent state space. It employs gradient-penalty regularization and hybrid-policy data training to mitigate value-function discontinuities and CBF estimation errors in the latent space. The method integrates Hamilton–Jacobi reachability analysis, latent-space modeling, gradient regularization, and reinforcement learning. Contribution/Results: LatentCBF achieves, for the first time, end-to-end smooth safety filtering compatible with CBF theory. In simulation and physical experiments, it doubles task completion rate (100% improvement) over conventional switching-based filters while maintaining high safety assurance and control smoothness.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Behavior Learning & Control
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling safe visuomotor control under hard-to-model constraints. However, existing methods implement "least-restrictive" filtering that discretely switch between nominal and safety policies, potentially undermining the task performance that makes modern visuomotor policies valuable. While reachability value functions can, in principle, be adapted to be control barrier functions (CBFs) for smooth optimization-based filtering, we theoretically and empirically show that current latent-space learning methods produce fundamentally incompatible value functions. We identify two sources of incompatibility: First, in HJ reachability, failures are encoded via a "margin function" in latent space, whose sign indicates whether or not a latent is in the constraint set. However, representing the margin function as a classifier yields saturated value functions that exhibit discontinuous jumps. We prove that the value function's Lipschitz constant scales linearly with the margin function's Lipschitz constant, revealing that smooth CBFs require smooth margins. Second, reinforcement learning (RL) approximations trained solely on safety policy data yield inaccurate value estimates for nominal policy actions, precisely where CBF filtering needs them. We propose the LatentCBF, which addresses both challenges through gradient penalties that lead to smooth margin functions without additional labeling, and a value-training procedure that mixes data from both nominal and safety policy distributions. Experiments on simulated benchmarks and hardware with a vision-based manipulation policy demonstrate that LatentCBF enables smooth safety filtering while doubling the task-completion rate over prior switching methods.

Problem

• Latent safety filters produce discontinuous value functions incompatible with smooth control barrier functions
• Classifier-based margin functions create saturated values with discontinuous jumps in latent space
• RL approximations yield inaccurate value estimates for nominal policy actions needed for CBF filtering

Innovation

• LatentCBF uses gradient penalties for smooth margin functions
• LatentCBF mixes nominal and safety policy data for training
• LatentCBF enables smooth optimization-based safety filtering