Problem
Research questions and friction points this paper is trying to address.
class imbalance
few-shot learning
synthetic data generation
visual recognition
data scarcity
Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks
🤖 AI Summary
This work addresses the challenge of class imbalance in domains such as medical imaging and industrial defect inspection, where positive (rare-class) samples are scarce. The authors propose a lightweight data augmentation approach leveraging pre-trained diffusion models, requiring only 20–50 real rare-class images to generate high-quality synthetic data through efficient fine-tuning of LoRA adapters. The study systematically investigates the optimal mixing ratio between synthetic and real data for downstream tasks. Notably, this is the first method to integrate LoRA-based fine-tuning with diffusion models for data augmentation, significantly improving recall and F1 scores for rare classes. Extensive experiments on chest X-ray and magnetic tile defect datasets demonstrate the method’s effectiveness and cross-domain scalability.
📝 Abstract
Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training.
We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only.
Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases.
These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Few-shot synthesis
Diffusion models
LoRA adaptation
Class imbalance
Data augmentation
🔎 Similar Papers
No similar papers found.
