Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability

2025-06-09

AI Summary

To address the vulnerability of deep learning models to adversarial attacks and the lack of reliable performance guarantees in high-stakes scenarios, this paper proposes the first robust and trustworthy framework integrating conformal prediction with adversarial training. We introduce Optimal-Size Perturbation Attack (OPSA) and its counterpart defense paradigm, OPSA-AT, which uniquely embeds conformal prediction’s statistical calibration and uncertainty quantification into the adversarial training pipeline—enabling uncertainty-driven robustness enhancement. Our method simultaneously ensures guaranteed coverage of prediction sets and strict statistical calibration while significantly improving model robustness against diverse adversarial attacks. Extensive evaluation on benchmarks including CIFAR-10 and CIFAR-100 demonstrates synergistic improvements in both robustness and generalization. This work establishes a novel paradigm for trustworthy AI deployment, offering rigorous theoretical guarantees alongside practical efficacy.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

As deep learning models are increasingly deployed in high-risk applications, robust defenses against adversarial attacks and reliable performance guarantees become paramount. Moreover, accuracy alone does not provide sufficient assurance or reliable uncertainty estimates for these models. This study advances adversarial training by leveraging principles from Conformal Prediction. Specifically, we develop an adversarial attack method, termed OPSA (OPtimal Size Attack), designed to reduce the efficiency of conformal prediction at any significance level by maximizing model uncertainty without requiring coverage guarantees. Correspondingly, we introduce OPSA-AT (Adversarial Training), a defense strategy that integrates OPSA within a novel conformal training paradigm. Experimental evaluations demonstrate that our OPSA attack method induces greater uncertainty compared to baseline approaches for various defenses. Conversely, our OPSA-AT defensive model significantly enhances robustness not only against OPSA but also other adversarial attacks, and maintains reliable prediction. Our findings highlight the effectiveness of this integrated approach for developing trustworthy and resilient deep learning models for safety-critical domains. Our code is available at https://github.com/bjbbbb/Enhancing-Adversarial-Robustness-with-Conformal-Prediction.

Problem

• Enhancing adversarial robustness using Conformal Prediction framework
• Developing OPSA attack to maximize model uncertainty efficiently
• Proposing OPSA-AT defense for improved robustness and reliable predictions

Innovation

• Leverages Conformal Prediction for robustness
• Introduces OPSA attack maximizing model uncertainty
• Develops OPSA-AT defense enhancing adversarial robustness