Beyond Confidence: Adaptive Abstention in Dual-Threshold Conformal Prediction for Autonomous System Perception

2025-02-11

AI Summary

This paper addresses the challenges of uncertainty quantification and reliable rejection in safety-critical perception systems operating under dynamic environmental conditions. We propose a dual-threshold conformal prediction framework that jointly optimizes a conformal confidence threshold—guaranteeing distribution-free statistical coverage ≥1−α—and an ROC-optimized rejection threshold—enabling risk-aware, adaptive abstention. To our knowledge, this is the first method to achieve synergistic co-optimization of both thresholds. Evaluated on image and LiDAR modalities across CIFAR-100, ImageNet1K, and ModelNet40, our approach achieves AUC scores of 0.993–0.995 and average coverage exceeding 90.0%. Under environmental degradation, rejection rates increase to 63.4%±0.5, while LiDAR-based coverage remains above 84.5%, significantly enhancing safety and controllability in high-risk applications such as autonomous driving.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Learning & Optimization for CV

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Safety-critical perception systems require both reliable uncertainty quantification and principled abstention mechanisms to maintain safety under diverse operational conditions. We present a novel dual-threshold conformalization framework that provides statistically-guaranteed uncertainty estimates while enabling selective prediction in high-risk scenarios. Our approach uniquely combines a conformal threshold ensuring valid prediction sets with an abstention threshold optimized through ROC analysis, providing distribution-free coverage guarantees (ge 1 - alpha) while identifying unreliable predictions. Through comprehensive evaluation on CIFAR-100, ImageNet1K, and ModelNet40 datasets, we demonstrate superior robustness across camera and LiDAR modalities under varying environmental perturbations. The framework achieves exceptional detection performance (AUC: 0.993 o0.995) under severe conditions while maintaining high coverage (>90.0%) and enabling adaptive abstention (13.5% o63.4%pm0.5) as environmental severity increases. For LiDAR-based perception, our approach demonstrates particularly strong performance, maintaining robust coverage (>84.5%) while appropriately abstaining from unreliable predictions. Notably, the framework shows remarkable stability under heavy perturbations, with detection performance (AUC: 0.995pm0.001) significantly outperforming existing methods across all modalities. Our unified approach bridges the gap between theoretical guarantees and practical deployment needs, offering a robust solution for safety-critical autonomous systems operating in challenging real-world conditions.

Problem

• Ensure safety in autonomous system perception
• Provide reliable uncertainty quantification
• Enable adaptive abstention in high-risk scenarios

Innovation

• Dual-threshold conformalization framework
• ROC-optimized abstention threshold
• Distribution-free coverage guarantees