Robust Semi-Supervised CT Radiomics for Lung Cancer Prognosis: Cost-Effective Learning with Limited Labels and SHAP Interpretation

2025-07-10

AI Summary

Label scarcity, poor generalizability, and limited interpretability hinder prognostic modeling from lung CT images. Method: We propose a robust radiomics framework integrating pseudo-labeling-based semi-supervised learning with SHAP-based interpretability analysis. Multi-scale features are extracted via LoG and wavelet filtering; 56 dimensionality reduction and 27 classification combinations are systematically evaluated to construct an end-to-end semi-supervised pipeline. Crucially, pseudo-labeling is innovatively coupled with SHAP to jointly enhance performance and clinical trustworthiness. Results: With only 10% labeled data, the model achieves 0.90 cross-validation accuracy and 0.88 external validation accuracy—outperforming the fully supervised baseline by 17% while significantly reducing variance. It demonstrates superior cross-center generalization, establishing an efficient, stable, and interpretable paradigm for small-sample medical image prognostication.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Background: CT imaging is vital for lung cancer management, offering detailed visualization for AI-based prognosis. However, supervised learning SL models require large labeled datasets, limiting their real-world application in settings with scarce annotations. Methods: We analyzed CT scans from 977 patients across 12 datasets extracting 1218 radiomics features using Laplacian of Gaussian and wavelet filters via PyRadiomics Dimensionality reduction was applied with 56 feature selection and extraction algorithms and 27 classifiers were benchmarked A semi supervised learning SSL framework with pseudo labeling utilized 478 unlabeled and 499 labeled cases Model sensitivity was tested in three scenarios varying labeled data in SL increasing unlabeled data in SSL and scaling both from 10 percent to 100 percent SHAP analysis was used to interpret predictions Cross validation and external testing in two cohorts were performed. Results: SSL outperformed SL, improving overall survival prediction by up to 17 percent. The top SSL model, Random Forest plus XGBoost classifier, achieved 0.90 accuracy in cross-validation and 0.88 externally. SHAP analysis revealed enhanced feature discriminability in both SSL and SL, especially for Class 1 survival greater than 4 years. SSL showed strong performance with only 10 percent labeled data, with more stable results compared to SL and lower variance across external testing, highlighting SSL's robustness and cost effectiveness. Conclusion: We introduced a cost-effective, stable, and interpretable SSL framework for CT-based survival prediction in lung cancer, improving performance, generalizability, and clinical readiness by integrating SHAP explainability and leveraging unlabeled data.

Problem

• Improving lung cancer prognosis with limited labeled CT scans
• Enhancing prediction accuracy using semi-supervised learning
• Reducing annotation costs while maintaining robust performance

Innovation

• Semi-supervised learning with pseudo-labeling
• Radiomics feature extraction via PyRadiomics
• SHAP analysis for prediction interpretability