HMIL: Hierarchical Multi-Instance Learning for Fine-Grained Whole Slide Image Classification

2024-11-12

IEEE Transactions on Medical Imaging

AI Summary

In fine-grained classification of whole-slide images (WSIs) for precision oncology, existing multiple instance learning (MIL) methods neglect hierarchical semantic relationships among pathological labels, limiting their ability to discriminate subtle morphological differences between tumor subtypes. To address this, we propose Hierarchical MIL (HMIL), a novel framework comprising: (1) a class-aware hierarchical attention mechanism that explicitly models the tree-structured dependencies among pathology labels; (2) supervised contrastive learning to enhance discriminability of fine-grained features; and (3) a curriculum-driven dynamic hierarchical weighting module that adaptively balances classification difficulty across hierarchical levels. Evaluated on CCC, BRACS, and PANDA datasets, HMIL achieves state-of-the-art performance, improving subtype classification accuracy by 3.2–5.7% over prior methods while demonstrating enhanced robustness and generalizability.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Fine-grained classification of whole slide images (WSIs) is essential in precision oncology, enabling precise cancer diagnosis and personalized treatment strategies. The core of this task involves distinguishing subtle morphological variations within the same broad category of gigapixel-resolution images, which presents a significant challenge. While the multi-instance learning (MIL) paradigm alleviates the computational burden of WSIs, existing MIL methods often overlook hierarchical label correlations, treating fine-grained classification as a flat multi-class classification task. To overcome these limitations, we introduce a novel hierarchical multi-instance learning (HMIL) framework. By facilitating on the hierarchical alignment of inherent relationships between different hierarchy of labels at instance and bag level, our approach provides a more structured and informative learning process. Specifically, HMIL incorporates a class-wise attention mechanism that aligns hierarchical information at both the instance and bag levels. Furthermore, we introduce supervised contrastive learning to enhance the discriminative capability for fine-grained classification and a curriculum-based dynamic weighting module to adaptively balance the hierarchical feature during training. Extensive experiments on our large-scale cytology cervical cancer (CCC) dataset and two public histology datasets, BRACS and PANDA, demonstrate the state-of-the-art class-wise and overall performance of our HMIL framework. Our source code is available at https://github.com/ChengJin-git/HMIL.

Problem

• Classify fine-grained whole slide images for cancer diagnosis
• Address hierarchical label correlations in multi-instance learning
• Improve discriminative capability via contrastive learning and dynamic weighting

Innovation

• Hierarchical multi-instance learning for fine-grained classification
• Class-wise attention mechanism aligning hierarchical information
• Supervised contrastive learning enhancing discriminative capability