TrustFed: Enabling Trustworthy Medical AI under Data Privacy Constraints

2026-03-23

AI Summary

This work addresses the challenges of cross-institutional medical AI model training—namely data heterogeneity, site-specific biases, and class imbalance—which degrade prediction reliability and undermine existing uncertainty quantification methods. To overcome these limitations without sharing raw patient data, the authors propose TrustFed, a federated framework that enables distribution-agnostic uncertainty quantification with finite-sample coverage guarantees. Central to TrustFed are two key innovations: a representation-aware client assignment mechanism and a soft nearest-neighbor threshold aggregation strategy. Evaluated on a large-scale clinical imaging dataset encompassing over 430,000 images across six modalities, TrustFed consistently delivers robust statistical coverage under diverse imbalance scenarios, marking a critical step toward clinically deployable, uncertainty-aware federated learning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Protecting patient privacy remains a fundamental barrier to scaling machine learning across healthcare institutions, where centralizing sensitive data is often infeasible due to ethical, legal, and regulatory constraints. Federated learning offers a promising alternative by enabling privacy-preserving, multi-institutional training without sharing raw patient data; however, real-world deployments face severe challenges from data heterogeneity, site-specific biases, and class imbalance, which degrade predictive reliability and render existing uncertainty quantification methods ineffective. Here, we present TrustFed, a federated uncertainty quantification framework that provides distribution-free, finite-sample coverage guarantees under heterogeneous and imbalanced healthcare data, without requiring centralized access. TrustFed introduces a representation-aware client assignment mechanism that leverages internal model representations to enable effective calibration across institutions, along with a soft-nearest threshold aggregation strategy that mitigates assignment uncertainty while producing compact and reliable prediction sets. Using over 430,000 medical images across six clinically distinct imaging modalities, we conduct one of the most comprehensive evaluations of uncertainty-aware federated learning in medical imaging, demonstrating robust coverage guarantees across datasets with diverse class cardinalities and imbalance regimes. By validating TrustFed at this scale and breadth, our study advances uncertainty-aware federated learning from proof-of-concept toward clinically meaningful, modality-agnostic deployment, positioning statistically guaranteed uncertainty as a core requirement for next-generation healthcare AI systems.

Problem

• federated learning
• data heterogeneity
• class imbalance
• uncertainty quantification
• medical AI

Innovation

• Federated Learning
• Uncertainty Quantification
• Medical Imaging
• Data Heterogeneity
• Privacy-Preserving AI