Robust Representation Consistency Model via Contrastive Denoising

2025-01-22

AI Summary

Existing diffusion-based classifiers suffer from poor robustness under large perturbations and high inference overhead. This paper proposes an implicit denoising-classification unification framework that shifts diffusion trajectory modeling from pixel-space generation to latent-space discrimination, and— for the first time—integrates instance-wise discrimination with temporal proximity representation alignment. The method enables robust classification via a single forward pass without explicit sampling. Leveraging contrastive learning and randomized smoothing, it enforces representation consistency along the diffusion trajectory and provides verifiable certified robustness. On ImageNet, our approach achieves an average certified accuracy improvement of 5.3% (up to 11.6% under large perturbations) while reducing inference cost by 85×, significantly improving the robustness-efficiency trade-off and establishing new state-of-the-art performance.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Data transparency and provenance

Abstract

Robustness is essential for deep neural networks, especially in security-sensitive applications. To this end, randomized smoothing provides theoretical guarantees for certifying robustness against adversarial perturbations. Recently, diffusion models have been successfully employed for randomized smoothing to purify noise-perturbed samples before making predictions with a standard classifier. While these methods excel at small perturbation radii, they struggle with larger perturbations and incur a significant computational overhead during inference compared to classical methods. To address this, we reformulate the generative modeling task along the diffusion trajectories in pixel space as a discriminative task in the latent space. Specifically, we use instance discrimination to achieve consistent representations along the trajectories by aligning temporally adjacent points. After fine-tuning based on the learned representations, our model enables implicit denoising-then-classification via a single prediction, substantially reducing inference costs. We conduct extensive experiments on various datasets and achieve state-of-the-art performance with minimal computation budget during inference. For example, our method outperforms the certified accuracy of diffusion-based methods on ImageNet across all perturbation radii by 5.3% on average, with up to 11.6% at larger radii, while reducing inference costs by 85$ imes$ on average. Codes are available at: https://github.com/jiachenlei/rRCM.

Problem

• Diffusion Models
• Image Classification
• Computational Cost

Innovation

• Improved Random Consistency Model
• Diffusion Path Transformation
• Enhanced Efficiency and Accuracy