Pattern-Calibrated Multimodal Prediction under Blockwise Missingness

2026-07-02

AI Summary

This work addresses the challenge of predicting outcomes under block-wise missingness in multimodal data, where observations follow specific patterns. The authors propose MOSAIC, a novel framework that disentangles shared and modality-specific representations to construct a base predictor using only the available information overlapping with the target observation pattern. To mitigate prediction bias, MOSAIC incorporates a cross-modal calibration mechanism that enables pattern-aware information borrowing without conflating heterogeneous modalities. Theoretical analysis delineates the conditions for effective cross-modal transfer and decomposes sources of prediction error. Extensive experiments on ICU mortality prediction, emotion recognition, and glaucoma classification demonstrate that MOSAIC significantly outperforms existing methods—particularly when target-pattern samples are scarce or inter-modal decision rules differ substantially.

Abstract

Blockwise missingness in multimodal data is usually treated as an incomplete-input problem. We instead focus on prediction for a prespecified observed-modality pattern, where the observed modality set determines the information on which the prediction rule can condition. A procedure that imputes missing modalities, zero-fills unobserved modalities, or trains a single pooled predictor may borrow information across patterns, but it can also mix pattern-specific prediction rules. We propose Multimodal Overlap-aware Shared-specific Alignment and Inter-pattern Calibration (MOSAIC), a pattern-calibrated framework for borrowing across missingness patterns without collapsing their prediction rules. MOSAIC learns shared and modality-specific representations, uses the available representations that overlap with the target pattern to fit a first-stage predictor, and then estimates the calibration gap from target-pattern data. We establish non-asymptotic bounds that decompose the error into overlap effective sample size, calibration gap, and representation-learning error, clarifying when cross-pattern borrowing improves over local fitting and when the improvement is controlled by rule mismatch or representation-learning error. Simulations examine representation recovery and target-pattern correction, and applications to ICU mortality prediction, emotion recognition, and glaucoma classification show gains when target-pattern samples are limited or pattern-specific rules differ.

Problem

• blockwise missingness
• multimodal prediction
• missingness patterns
• pattern-specific prediction
• incomplete multimodal data

Innovation

• blockwise missingness
• multimodal learning
• pattern-calibrated prediction
• representation alignment
• inter-pattern calibration