Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports

2025-04-16

AI Summary

This work addresses the challenge of jointly ensuring safety and preserving human autonomy in AI-assisted racing. We propose a Human-Centered Safety Filter (HCSF), whose core innovation is the Quality-Control Barrier Function (Q-CBF)—the first model-free safety filter enabling smooth, minimally intrusive safety constraints within a high-fidelity black-box simulator (Assetto Corsa). Integrating a neural safety value function with interactive learning, HCSF is the first framework to jointly quantify safety, human agency, and subjective comfort in shared driving. User studies demonstrate that, compared to unassisted driving, HCSF significantly reduces collision rates while improving user satisfaction. Relative to conventional safety filters, HCSF achieves comparable safety robustness while substantially enhancing human agency, ride comfort, and overall user satisfaction.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Security and Privacy: Large-scale security measurements

Abstract

We propose a human-centered safety filter (HCSF) for shared autonomy that significantly enhances system safety without compromising human agency. Our HCSF is built on a neural safety value function, which we first learn scalably through black-box interactions and then use at deployment to enforce a novel quality control barrier function (Q-CBF) safety constraint. Since this Q-CBF safety filter does not require any knowledge of the system dynamics for both synthesis and runtime safety monitoring and intervention, our method applies readily to complex, black-box shared autonomy systems. Notably, our HCSF's CBF-based interventions modify the human's actions minimally and smoothly, avoiding the abrupt, last-moment corrections delivered by many conventional safety filters. We validate our approach in a comprehensive in-person user study using Assetto Corsa-a high-fidelity car racing simulator with black-box dynamics-to assess robustness in"driving on the edge"scenarios. We compare both trajectory data and drivers' perceptions of our HCSF assistance against unassisted driving and a conventional safety filter. Experimental results show that 1) compared to having no assistance, our HCSF improves both safety and user satisfaction without compromising human agency or comfort, and 2) relative to a conventional safety filter, our proposed HCSF boosts human agency, comfort, and satisfaction while maintaining robustness.

Problem

• Enhancing system safety without compromising human agency
• Applying safety filter to black-box shared autonomy systems
• Improving safety and user satisfaction in high-risk scenarios

Innovation

• Neural safety value function for scalable learning
• Quality control barrier function for safety enforcement
• Minimal and smooth human action modifications