Comparing ImageNet Pre-training with Digital Pathology Foundation Models for Whole Slide Image-Based Survival Analysis

2024-05-20

AI Summary

Digital pathology foundation models (e.g., UNI, Hibou) show promise for whole-slide image (WSI) survival analysis, yet their transfer efficacy relative to generic vision pretraining (e.g., ImageNet-trained ResNet50) remains quantitatively unverified. Method: We conduct a systematic evaluation under a unified multi-instance learning (MIL) framework, comparing backbone architectures—including pathology-specific and generic models—on WSI-based survival prediction. We further investigate ensemble strategies combining pathology foundation models. Contribution/Results: Pathology-specific foundation models significantly outperform ImageNet-pretrained ResNet50, yielding consistent improvements in C-index and classification accuracy. Even a single pathology model surpasses the generic baseline; ensembling further boosts performance, though marginal gains diminish with increasing MIL architectural complexity. To our knowledge, this is the first systematic, quantitative study validating the superiority of pathology foundation models in WSI survival analysis. Code will be publicly released.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Computer Vision: Other Foundations of Computer Vision
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The abundance of information present in Whole Slide Images (WSIs) renders them an essential tool for survival analysis. Several Multiple Instance Learning frameworks proposed for this task utilize a ResNet50 backbone pre-trained on natural images. By leveraging recenetly released histopathological foundation models such as UNI and Hibou, the predictive prowess of existing MIL networks can be enhanced. Furthermore, deploying an ensemble of digital pathology foundation models yields higher baseline accuracy, although the benefits appear to diminish with more complex MIL architectures. Our code will be made publicly available upon acceptance.

Problem

• Enhance survival analysis using histopathological foundation models
• Compare ImageNet pre-training with digital pathology models
• Evaluate ensemble models' impact on MIL framework accuracy

Innovation

• Leveraging histopathological foundation models UNI and Hibou
• Enhancing MIL networks with digital pathology models
• Ensemble of models improves baseline accuracy