A deep learning pipeline for PAM50 subtype classification using histopathology images and multi-objective patch selection

2026-04-02

AI Summary

This study addresses the challenge of reducing reliance on costly molecular assays by directly predicting PAM50 molecular subtypes of breast cancer from hematoxylin and eosin (H&E)–stained whole-slide images. To this end, the authors propose a multi-objective optimization–driven deep learning framework that integrates the NSGA-II algorithm with Monte Carlo Dropout–based uncertainty estimation to jointly optimize patch selection across criteria including informativeness, spatial diversity, predictive uncertainty, and computational efficiency. The model employs a ResNet18 backbone for feature extraction coupled with a custom CNN-based classification head. Evaluated on the TCGA-BRCA dataset, it achieves an F1-score of 0.8812 and an AUC of 0.9841, and demonstrates robust generalization on the external CPTAC-BRCA validation set with an F1-score of 0.7952 and an AUC of 0.9512, significantly enhancing both predictive performance and computational efficiency.

Abstract

Breast cancer is a highly heterogeneous disease with diverse molecular profiles. The PAM50 gene signature is widely recognized as a standard for classifying breast cancer into intrinsic subtypes, enabling more personalized treatment strategies. In this study, we introduce a novel optimization-driven deep learning framework that aims to reduce reliance on costly molecular assays by directly predicting PAM50 subtypes from H&E-stained whole-slide images (WSIs). Our method jointly optimizes patch informativeness, spatial diversity, uncertainty, and patch count by combining the non-dominated sorting genetic algorithm II (NSGA-II) with Monte Carlo dropout-based uncertainty estimation. The proposed method can identify a small but highly informative patch subset for classification. We used a ResNet18 backbone for feature extraction and a custom CNN head for classification. For evaluation, we used the internal TCGA-BRCA dataset as the training cohort and the external CPTAC-BRCA dataset as the test cohort. On the internal dataset, an F1-score of 0.8812 and an AUC of 0.9841 using 627 WSIs from the TCGA-BRCA cohort were achieved. The performance of the proposed approach on the external validation dataset showed an F1-score of 0.7952 and an AUC of 0.9512. These findings indicate that the proposed optimization-guided, uncertainty-aware patch selection can achieve high performance and improve the computational efficiency of histopathology-based PAM50 classification compared to existing methods, suggesting a scalable imaging-based replacement that has the potential to support clinical decision-making.

Problem

• PAM50 subtype classification
• histopathology images
• breast cancer
• H&E-stained whole-slide images
• molecular assays

Innovation

• multi-objective optimization
• uncertainty-aware patch selection
• PAM50 classification
• deep learning
• histopathology