Robust Classification by Coupling Data Mollification with Label Smoothing

2024-06-03

arXiv.org

AI Summary

To address the degradation in model robustness and miscalibrated uncertainty estimation under test-time image corruptions (e.g., noise, blur), this paper proposes a degradation-aware joint training framework. It uniquely couples generative diffusion-inspired data denoising—comprising Gaussian noise injection and Gaussian blurring—with label smoothing, thereby implicitly aligning degradation severity with predictive confidence during standard training. The method incurs no additional inference overhead and seamlessly integrates with existing data augmentation pipelines. Evaluated across multiple image corruption benchmarks on CIFAR-10/100, TinyImageNet, and ImageNet, it consistently improves classification robustness (average +2.1% top-1 accuracy) and uncertainty calibration quality (reducing Expected Calibration Error by up to 38%). This yields an efficient, plug-and-play training paradigm for degradation-robust deep learning models.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Introducing training-time augmentations is a key technique to enhance generalization and prepare deep neural networks against test-time corruptions. Inspired by the success of generative diffusion models, we propose a novel approach of coupling data mollification, in the form of image noising and blurring, with label smoothing to align predicted label confidences with image degradation. The method is simple to implement, introduces negligible overheads, and can be combined with existing augmentations. We demonstrate improved robustness and uncertainty quantification on the corrupted image benchmarks of CIFAR, TinyImageNet and ImageNet datasets.

Problem

• Enhancing robustness against test-time image corruptions
• Aligning label confidences with degraded image quality
• Improving uncertainty quantification in corrupted image benchmarks

Innovation

• Couples data mollification with label smoothing
• Uses image noising and blurring techniques
• Combines with existing augmentations simply