๐ค AI Summary
This work addresses the vulnerability of machine learning components in safety-critical systems to long-tail failures, a challenge inadequately mitigated by conventional runtime assurance architectures that struggle to balance safety and performance. To overcome this limitation, the paper proposes a cooperative simplex architecture that bidirectionally integrates a machine learning component with a safety monitor, thereby enhancing overall system performance while preserving formal safety guarantees. Crucially, the architecture enables the monitor to leverage the machine learning outputโa departure from traditional simplex designsโand rigorously derives integration conditions sufficient for ensuring safety. By synergistically combining control- and perception-oriented simplex design, formal verification, and machine learning techniques, the approach yields a provably safe cooperative assurance system. Empirical validation on an autonomous driving obstacle detection task demonstrates that the method significantly improves performance without compromising stringent safety assurances.
๐ Abstract
Autonomous systems increasingly rely on machine-learning (ML) components for safety-critical tasks such as perception and control in autonomous vehicles (AVs). While ML enables essential capabilities, it inevitably exhibits long-tail faults that make it unsuitable for safety-critical tasks. Runtime assurance (RTA) mitigates this issue by pairing ML components with verifiable safety monitors, e.g., Control Simplex and Perception Simplex architectures. However, the limited performance of safety monitors remains a major bottleneck.
The Synergistic Simplex (SS) architecture improves system performance by enabling bidirectional integration between ML components and safety monitors while preserving formal safety guarantees. The key innovation here is allowing safety monitors to use ML outputs, which is typically prohibited in RTA systems. We formally derive conditions under which this integration preserves safety and demonstrate the performance benefits. We present the design, analysis, and evaluation of SS for AV obstacle detection.