SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentation

2025-05-17

AI Summary

Addressing the core challenge in domain-specific image classification—where data augmentation struggles to simultaneously ensure diversity, fidelity, and label clarity—this paper proposes the first synergistic framework integrating saliency-guided mixing with fine-tuned diffusion models. Our method localizes semantically critical regions via saliency detection, designs a foreground-preserving diffusion model fine-tuning strategy, and introduces a mixing mechanism explicitly regularized by label consistency constraints, thereby jointly optimizing the three objectives. This formulation mitigates inherent diffusion-model limitations, including noise sensitivity and structural distortion. Extensive experiments demonstrate systematic superiority over state-of-the-art methods across four challenging settings: fine-grained recognition, long-tailed distribution learning, few-shot classification, and background robustness.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Data augmentation for domain-specific image classification tasks often struggles to simultaneously address diversity, faithfulness, and label clarity of generated data, leading to suboptimal performance in downstream tasks. While existing generative diffusion model-based methods aim to enhance augmentation, they fail to cohesively tackle these three critical aspects and often overlook intrinsic challenges of diffusion models, such as sensitivity to model characteristics and stochasticity under strong transformations. In this paper, we propose a novel framework that explicitly integrates diversity, faithfulness, and label clarity into the augmentation process. Our approach employs saliency-guided mixing and a fine-tuned diffusion model to preserve foreground semantics, enrich background diversity, and ensure label consistency, while mitigating diffusion model limitations. Extensive experiments across fine-grained, long-tail, few-shot, and background robustness tasks demonstrate our method's superior performance over state-of-the-art approaches.

Problem

• Address diversity, faithfulness, label clarity in image augmentation
• Overcome diffusion models' sensitivity and stochasticity issues
• Enhance domain-specific classification with saliency-guided mixing

Innovation

• Saliency-guided mixing preserves foreground semantics
• Fine-tuned diffusion model enriches background diversity
• Label consistency ensured during augmentation process