Do Not Forget the Obvious - RISC: A Risk-Informed Slice-Coverage Protocol for Safe Autonomous Driving

2026-08-12

AI Summary

Current autonomous driving evaluation methods rely on aggregate metrics that struggle to effectively capture system failures in high-risk, low-frequency scenarios. To address this limitation, this work proposes RISC—a risk-informed evaluation protocol that enables model-agnostic and interpretable stress testing through computable risk slices, lightweight data annotation, and risk-guided sampling. Furthermore, the framework leverages large language models to assist in identifying critical yet often overlooked scenarios. Evaluated on monocular pedestrian perception tasks, RISC dramatically improves the detection rate of critical failures from 34.0% to 98.5%, demonstrating its superior capability in efficiently uncovering high-risk system deficiencies.

Abstract

Aggregate metrics may not fully reflect performance in insufficiently examined high-risk driving conditions. We propose RISC (Risk-Informed Slice Coverage), a practical protocol for risk-guided stress testing and coverage-qualified evaluation. Risk-guided stress testing directs a finite audit budget toward risk-relevant sub-datasets, called risk slices, while coverage-qualified evaluation reports results together with explicit statements about which slices are sufficiently or insufficiently covered. The protocol translates safety concerns into machine-readable risk slices, uses lightweight signals to tag candidate data, selects a compact audit set by risk, and qualifies the results using coverage evidence. An LLM can optionally support this process by surfacing relevant but potentially overlooked conditions during test planning, thereby helping engineers not to forget the obvious. RISC is model-agnostic and can be applied to perception modules, driving models, and other autonomous-driving subsystems. We instantiate the protocol for monocular pedestrian perception using 1,000 frames from the Zenseact Open Dataset, image statistics, and a YOLO-based detector proxy. In this proof-of-concept study, risk-guided selection increases critical failure discovery from 34.0% under random sampling to 98.5%. RISC provides a lightweight, assurance-oriented evaluation layer that complements scenario categorization, coverage assessment, and broader testing-and-verification workflows.

Problem

• autonomous driving
• risk assessment
• coverage evaluation
• stress testing
• safety validation

Innovation

• Risk-Informed Testing
• Slice Coverage
• Autonomous Driving Safety
• Model-Agnostic Evaluation
• Stress Testing