Sample Selection Bias in Machine Learning for Healthcare

2024-05-13

arXiv.org

AI Summary

Medical machine learning models frequently suffer from poor generalization to target clinical populations due to sample selection bias (SSB), posing significant clinical risks. While existing distributional correction methods mitigate SSB, they often degrade predictive performance. To address this, we propose a novel “target-identification-first” paradigm that abandons conventional distribution balancing in favor of precise identification and modeling of target subpopulations. Our approach employs a dual-network architecture: T-Net performs unsupervised target subpopulation identification, while MT-Net jointly optimizes identification and prediction via multi-task learning. Integrating deep neural networks, synthetic/semi-synthetic data augmentation, and subpopulation-aware mechanisms, our method achieves statistically significant improvements over state-of-the-art bias-correction techniques across multiple real-world healthcare benchmarks. It effectively mitigates SSB-induced performance degradation, enhancing model generalizability, robustness, and clinical applicability on target populations.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Bias, Fairness & Privacy
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

While machine learning algorithms hold promise for personalised medicine, their clinical adoption remains limited, partly due to biases that can compromise the reliability of predictions. In this paper, we focus on sample selection bias (SSB), a specific type of bias where the study population is less representative of the target population, leading to biased and potentially harmful decisions. Despite being well-known in the literature, SSB remains scarcely studied in machine learning for healthcare. Moreover, the existing machine learning techniques try to correct the bias mostly by balancing distributions between the study and the target populations, which may result in a loss of predictive performance. To address these problems, our study illustrates the potential risks associated with SSB by examining SSB's impact on the performance of machine learning algorithms. Most importantly, we propose a new research direction for addressing SSB, based on the target population identification rather than the bias correction. Specifically, we propose two independent networks(T-Net) and a multitasking network (MT-Net) for addressing SSB, where one network/task identifies the target subpopulation which is representative of the study population and the second makes predictions for the identified subpopulation. Our empirical results with synthetic and semi-synthetic datasets highlight that SSB can lead to a large drop in the performance of an algorithm for the target population as compared with the study population, as well as a substantial difference in the performance for the target subpopulations that are representative of the selected and the non-selected patients from the study population. Furthermore, our proposed techniques demonstrate robustness across various settings, including different dataset sizes, event rates, and selection rates, outperforming the existing bias correction techniques.

Problem

• Addressing sample selection bias in healthcare machine learning
• Proposing target population identification over bias correction
• Developing T-Net and MT-Net networks for SSB mitigation

Innovation

• Two independent networks for target identification
• Multitasking network addressing sample selection bias
• Target population identification over bias correction