HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation

2025-11-18

AI Summary

Medical image segmentation is often hindered by high annotation costs, limited labeled data, and consequent overfitting. Existing self-supervised or semi-supervised approaches rely on handcrafted pretext tasks or high-quality pseudo-labels—introducing complexity—while conventional data augmentation fails to preserve local semantic consistency in segmentation. To address these limitations, we propose HSMix, a plug-and-play augmentation method that jointly leverages superpixel segmentation and saliency-aware brightness mixing. HSMix synergistically combines hard mixing (splicing homogeneous superpixel regions) and soft mixing (salience-guided local illumination adjustment), thereby preserving semantic boundary integrity while enhancing data diversity. Crucially, it augments both images and corresponding masks simultaneously without requiring architectural modifications. Evaluated across multi-modal medical image segmentation benchmarks, HSMix consistently improves generalization performance. It is model-agnostic, computationally efficient, and broadly applicable across diverse segmentation architectures and imaging modalities.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance

Abstract

Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images. Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in https://github.com/DanielaPlusPlus/HSMix.

Problem

• Addresses data scarcity in medical image segmentation tasks
• Explores local image editing augmentation for segmentation effectiveness
• Proposes hard and soft mixing to preserve semantic information

Innovation

• HSMix combines superpixels from two source images
• Soft mixing adjusts brightness using saliency coefficients
• Plug-and-play augmentation preserves local semantic information