GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning

2026-04-28

AI Summary

This work addresses the challenge of unsupervised modality imputation in distributed multimodal learning, where clients often lack certain modalities and existing methods struggle to effectively leverage observed modalities for completion. To this end, we introduce graph neural networks (GNNs) into the Patchwork Learning framework—marking the first such integration—and propose a flexible, unsupervised approach for fusing heterogeneous observed modalities. By modeling inter-client relationships through GNNs, our method fully exploits all available modality information, significantly enhancing the robustness and accuracy of imputation under noisy conditions. Experimental results demonstrate that the proposed approach achieves state-of-the-art performance on standard multimodal benchmarks and exhibits superior disease prediction capability on real-world distributed electronic health record data.

Abstract

Current research on distributed multi-modal learning typically assumes that clients can access complete information across all modalities, which may not hold in practice. In this paper, we explore patchwork learning, in which the modalities available to different clients vary, and the objective is to impute the missing modalities for each client in an unsupervised manner. Existing methods are shown not to fully utilize the modality information as they tend to rely on only a subset of the observed modalities. To address this issue, we propose GraphPL, which combines graph neural networks with patchwork learning to flexibly integrate all observed modalities and remains robust with noisy inputs. Experimental results show that GraphPL achieves SOTA performance on benchmark datasets. Our results on real-world distributed electronic health record dataset show GraphPL learns strong downstream features and enables tasks like disease prediction via superior modality imputation.

Problem

• modality imputation
• patchwork learning
• distributed multimodal learning
• missing modalities
• unsupervised imputation

Innovation

• Graph Neural Networks
• Patchwork Learning
• Modality Imputation
• Distributed Multi-modal Learning
• Unsupervised Learning