How Effective Can Dropout Be in Multiple Instance Learning ?

2025-04-21

AI Summary

To address feature noise and weak-supervision generalization bottlenecks in multiple instance learning (MIL) for whole-slide image (WSI) classification, this paper presents the first systematic investigation of Dropout’s role in MIL and proposes MIL-Dropout: a novel technique that dynamically discards the most discriminative instances—prior to bag-level aggregation—based on instance importance, thereby enhancing model robustness against feature noise and label uncertainty. Integrated into a two-stage MIL framework, MIL-Dropout requires no architectural modifications to the backbone network and incurs negligible computational overhead. Extensive experiments across five standard MIL benchmarks and two WSI datasets demonstrate consistent performance improvements over state-of-the-art MIL models, validating its generalizability and practical efficacy. The implementation is publicly available.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multiple Instance Learning (MIL) is a popular weakly-supervised method for various applications, with a particular interest in histological whole slide image (WSI) classification. Due to the gigapixel resolution of WSI, applications of MIL in WSI typically necessitate a two-stage training scheme: first, extract features from the pre-trained backbone and then perform MIL aggregation. However, it is well-known that this suboptimal training scheme suffers from"noisy"feature embeddings from the backbone and inherent weak supervision, hindering MIL from learning rich and generalizable features. However, the most commonly used technique (i.e., dropout) for mitigating this issue has yet to be explored in MIL. In this paper, we empirically explore how effective the dropout can be in MIL. Interestingly, we observe that dropping the top-k most important instances within a bag leads to better performance and generalization even under noise attack. Based on this key observation, we propose a novel MIL-specific dropout method, termed MIL-Dropout, which systematically determines which instances to drop. Experiments on five MIL benchmark datasets and two WSI datasets demonstrate that MIL-Dropout boosts the performance of current MIL methods with a negligible computational cost. The code is available at https://github.com/ChongQingNoSubway/MILDropout.

Problem

• Effectiveness of dropout in Multiple Instance Learning
• Noisy feature embeddings in WSI classification
• Proposing MIL-Dropout for better performance

Innovation

• Drop top-k instances for better performance
• Propose MIL-Dropout for systematic dropping
• Boost MIL methods with low computational cost