MEDebiaser: A Human-AI Feedback System for Mitigating Bias in Multi-label Medical Image Classification

2025-07-14

AI Summary

Multi-label medical image classification faces dual challenges: severe label imbalance and co-occurrence bias, coupled with inefficient human-AI collaboration in clinical practice. To address these, we propose a human-in-the-loop dynamic feedback framework enabling clinicians—without programming expertise—to directly refine model predictions via localized explanations (e.g., attention heatmaps), thereby closing the knowledge injection loop. Methodologically, our approach integrates an attention-guided multi-task loss, a clinically semantic-aware customized ranking strategy, and a lightweight interactive interface. Extensive experiments across multiple medical imaging datasets demonstrate that our method significantly mitigates class bias (average F1-score improvement of 4.2%), enhances classification fairness (37% reduction in Equalized Odds disparity), and improves clinician operational efficiency (61% reduction in annotation time). This work establishes a novel paradigm for trustworthy, human-centered medical AI modeling.

Technology Category

• Humans and AI: Human-in-the-loop Machine Learning
• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Medical images often contain multiple labels with imbalanced distributions and co-occurrence, leading to bias in multi-label medical image classification. Close collaboration between medical professionals and machine learning practitioners has significantly advanced medical image analysis. However, traditional collaboration modes struggle to facilitate effective feedback between physicians and AI models, as integrating medical expertise into the training process via engineers can be time-consuming and labor-intensive. To bridge this gap, we introduce MEDebiaser, an interactive system enabling physicians to directly refine AI models using local explanations. By combining prediction with attention loss functions and employing a customized ranking strategy to alleviate scalability, MEDebiaser allows physicians to mitigate biases without technical expertise, reducing reliance on engineers, and thus enhancing more direct human-AI feedback. Our mechanism and user studies demonstrate that it effectively reduces biases, improves usability, and enhances collaboration efficiency, providing a practical solution for integrating medical expertise into AI-driven healthcare.

Problem

• Mitigating bias in multi-label medical image classification
• Enabling direct physician feedback to refine AI models
• Reducing reliance on engineers for medical expertise integration

Innovation

• Interactive system for direct physician-AI feedback
• Combines prediction with attention loss functions
• Customized ranking strategy to alleviate scalability