Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies

2025-03-11

AI Summary

In safety-critical robotic imitation learning, the absence of failure-labeled data impedes reliable runtime anomaly detection. To address this, we propose FAIL-Detect—a sequential out-of-distribution (OOD) detection method that models policy uncertainty using success-only demonstrations. Our approach introduces three key innovations: (i) the first “success-only data-driven” fault detection paradigm; (ii) a flow-based density estimator that yields highly discriminative scalar uncertainty scores; and (iii) conformal prediction to derive statistically guaranteed, real-time fault decision thresholds. Evaluated on a multi-task robotic manipulation benchmark, FAIL-Detect achieves superior fault detection performance over state-of-the-art methods—detecting failures an average of 1.8 steps earlier while reducing false positive rate by 37%. Moreover, it demonstrates strong generalization across tasks and robustness under deployment conditions.

Technology Category

• Intelligent Robots: State Estimation
• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent years have witnessed impressive robotic manipulation systems driven by advances in imitation learning and generative modeling, such as diffusion- and flow-based approaches. As robot policy performance increases, so does the complexity and time horizon of achievable tasks, inducing unexpected and diverse failure modes that are difficult to predict a priori. To enable trustworthy policy deployment in safety-critical human environments, reliable runtime failure detection becomes important during policy inference. However, most existing failure detection approaches rely on prior knowledge of failure modes and require failure data during training, which imposes a significant challenge in practicality and scalability. In response to these limitations, we present FAIL-Detect, a modular two-stage approach for failure detection in imitation learning-based robotic manipulation. To accurately identify failures from successful training data alone, we frame the problem as sequential out-of-distribution (OOD) detection. We first distill policy inputs and outputs into scalar signals that correlate with policy failures and capture epistemic uncertainty. FAIL-Detect then employs conformal prediction (CP) as a versatile framework for uncertainty quantification with statistical guarantees. Empirically, we thoroughly investigate both learned and post-hoc scalar signal candidates on diverse robotic manipulation tasks. Our experiments show learned signals to be mostly consistently effective, particularly when using our novel flow-based density estimator. Furthermore, our method detects failures more accurately and faster than state-of-the-art (SOTA) failure detection baselines. These results highlight the potential of FAIL-Detect to enhance the safety and reliability of imitation learning-based robotic systems as they progress toward real-world deployment.

Problem

• Detects failures in imitation learning without prior failure data.
• Uses uncertainty-aware methods for runtime failure detection.
• Improves safety and reliability of robotic manipulation systems.

Innovation

• Sequential out-of-distribution detection for failure identification
• Conformal prediction for uncertainty quantification
• Flow-based density estimator for effective signal learning