Uncertainty-Aware Generative Oversampling Using an Entropy-Guided Conditional Variational Autoencoder

2025-09-29

AI Summary

Traditional oversampling methods (e.g., SMOTE) fail on high-dimensional biomedical data exhibiting both class imbalance and intrinsic nonlinear manifold structure, while existing generative models (e.g., CVAE) neglect uncertainty in boundary-region samples. Method: We propose a local entropy-guided generative oversampling framework that, for the first time, incorporates Shannon’s local entropy as a boundary uncertainty metric into the CVAE architecture. We design an entropy-weighted loss (LEWL) and an entropy-guided sampling strategy to focus modeling and robust synthesis on high-uncertainty regions. Contribution/Results: Experiments on ADNI and TCGA lung cancer genomic datasets demonstrate significant improvements in minority-class classification performance over SMOTE, ADASYN, and standard CVAE. Our method achieves superior generalizability and sample plausibility in complex nonlinear settings, validating its effectiveness in capturing manifold-aware, uncertainty-informed synthetic data generation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Class imbalance remains a major challenge in machine learning, especially for high-dimensional biomedical data where nonlinear manifold structures dominate. Traditional oversampling methods such as SMOTE rely on local linear interpolation, often producing implausible synthetic samples. Deep generative models like Conditional Variational Autoencoders (CVAEs) better capture nonlinear distributions, but standard variants treat all minority samples equally, neglecting the importance of uncertain, boundary-region examples emphasized by heuristic methods like Borderline-SMOTE and ADASYN. We propose Local Entropy-Guided Oversampling with a CVAE (LEO-CVAE), a generative oversampling framework that explicitly incorporates local uncertainty into both representation learning and data generation. To quantify uncertainty, we compute Shannon entropy over the class distribution in a sample's neighborhood: high entropy indicates greater class overlap, serving as a proxy for uncertainty. LEO-CVAE leverages this signal through two mechanisms: (i) a Local Entropy-Weighted Loss (LEWL) that emphasizes robust learning in uncertain regions, and (ii) an entropy-guided sampling strategy that concentrates generation in these informative, class-overlapping areas. Applied to clinical genomics datasets (ADNI and TCGA lung cancer), LEO-CVAE consistently improves classifier performance, outperforming both traditional oversampling and generative baselines. These results highlight the value of uncertainty-aware generative oversampling for imbalanced learning in domains governed by complex nonlinear structures, such as omics data.

Problem

• Addresses class imbalance in high-dimensional biomedical data with nonlinear structures
• Improves generative oversampling by incorporating local uncertainty into representation learning
• Focuses synthetic sample generation on uncertain boundary regions between classes

Innovation

• Uses entropy-guided conditional variational autoencoder for oversampling
• Incorporates local uncertainty into representation learning and generation
• Employs entropy-weighted loss and sampling for class-overlapping regions