Using Synthetic Images to Augment Small Medical Image Datasets

2025-03-02

AI Summary

To address the performance limitations of deep learning models under scarce medical imaging data, this paper proposes an enhanced conditional StyleGAN2 framework for controllable generation of high-resolution, multi-modal medical images, balancing anatomical fidelity and cross-modal consistency. Methodologically, it integrates medical-prior-guided conditional inputs, adaptive normalization optimization, and modality-aware loss design, coupled with U-Net-based segmentation and quantitative evaluation using FID and LPIPS. Experiments demonstrate high visual realism in generated images; however, downstream semantic segmentation performance shows only marginal improvement, revealing a critical bottleneck: limited generalizability of synthetic data for clinical tasks. This finding provides key empirical insights and methodological reflections for synthetic-data-driven medical AI, highlighting the gap between perceptual quality and task-relevant utility in generative modeling for healthcare applications.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent years have witnessed a growing academic and industrial interest in deep learning (DL) for medical imaging. To perform well, DL models require very large labeled datasets. However, most medical imaging datasets are small, with a limited number of annotated samples. The reason they are small is usually because delineating medical images is time-consuming and demanding for oncologists. There are various techniques that can be used to augment a dataset, for example, to apply affine transformations or elastic transformations to available images, or to add synthetic images generated by a Generative Adversarial Network (GAN). In this work, we have developed a novel conditional variant of a current GAN method, the StyleGAN2, to generate multi-modal high-resolution medical images with the purpose to augment small medical imaging datasets with these synthetic images. We use the synthetic and real images from six datasets to train models for the downstream task of semantic segmentation. The quality of the generated medical images and the effect of this augmentation on the segmentation performance were evaluated afterward. Finally, the results indicate that the downstream segmentation models did not benefit from the generated images. Further work and analyses are required to establish how this augmentation affects the segmentation performance.

Problem

• Addressing small medical image datasets for deep learning
• Generating synthetic images to augment limited annotated samples
• Evaluating synthetic image impact on semantic segmentation performance

Innovation

• Developed conditional StyleGAN2 for medical images
• Generated multi-modal high-resolution synthetic images
• Evaluated synthetic images' impact on segmentation