One Sample is Enough to Make Conformal Prediction Robust

2025-06-19

AI Summary

Existing Robust Conformal Prediction (RCP) methods rely on multiple random smoothing samples, incurring prohibitive computational overhead. Method: We propose RCP1, the first single-sample RCP framework that transfers robustness certification from pointwise prediction scores to the conformal quantile estimation process itself—enabling robust prediction set construction with only one forward pass. RCP1 integrates binary robustness certificates with randomized smoothing, requiring neither model gradients nor structural assumptions, and uniformly supports both classification and regression. Contribution/Results: Under rigorous statistical coverage guarantees, RCP1 substantially reduces average prediction set size while cutting computational cost by ∼99% compared to state-of-the-art methods using 100 smoothing samples—achieving a 100× speedup. This establishes a new paradigm for high-throughput robust inference.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Given any model, conformal prediction (CP) returns prediction sets guaranteed to include the true label with high adjustable probability. Robust CP (RCP) extends this to inputs with worst-case noise. A well-established approach is to use randomized smoothing for RCP since it is applicable to any black-box model and provides smaller sets compared to deterministic methods. However, current smoothing-based RCP requires many model forward passes per each input which is computationally expensive. We show that conformal prediction attains some robustness even with a forward pass on a single randomly perturbed input. Using any binary certificate we propose a single sample robust CP (RCP1). Our approach returns robust sets with smaller average set size compared to SOTA methods which use many (e.g. around 100) passes per input. Our key insight is to certify the conformal prediction procedure itself rather than individual scores. Our approach is agnostic to the setup (classification and regression). We further extend our approach to smoothing-based robust conformal risk control.

Problem

• Reducing computational cost in robust conformal prediction
• Achieving robustness with single perturbed input sample
• Certifying conformal prediction procedure instead of individual scores

Innovation

• Uses single sample robust CP (RCP1)
• Certifies conformal prediction procedure
• Applicable to classification and regression