General Incomplete Multimodal Learning via Dynamic Quality Perception

2026-07-07

AI Summary

This work addresses the challenge of learning from multimodal data in real-world scenarios, where modalities often suffer simultaneously from complete absence and severe internal degradation—a mixed missingness problem that existing methods struggle to handle effectively. To tackle this, we propose GIML, a general framework for incomplete multimodal learning that, for the first time, unifies modality missingness and degradation as a continuous information deterioration process. GIML incorporates a noise-aware quality estimator and a noise–semantics disentanglement module, leveraging dynamic quality-awareness, noise-injected training, and adaptive fusion to achieve strong generalization across unseen degradation patterns. Extensive experiments demonstrate that GIML consistently outperforms state-of-the-art approaches across diverse modalities and datasets, exhibiting remarkable robustness and broad applicability.

Abstract

Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.

Problem

• incomplete multimodal learning
• inter-modality missing
• intra-modality degradation
• modality corruption
• multimodal robustness

Innovation

• General Incomplete Multimodal Learning
• Dynamic Quality Perception
• Intra-modality Degradation
• Noise-aware Quality Estimator
• Noise-Semantic Decoupling