MIRRAMS: Towards Training Models Robust to Missingness Distribution Shifts

2025-07-10

AI Summary

In real-world settings, distributional shifts in missingness patterns between training and test data severely compromise model robustness. To address this, we propose MIRRAMS—a novel framework that establishes the first theoretical conditions for missingness-robust learning. MIRRAMS introduces a joint objective combining mutual information regularization and consistency regularization, enabling adaptive modeling of unseen missingness patterns without prior knowledge of their distribution. The method is inherently compatible with semi-supervised learning and exhibits plug-and-play modularity. Extensive experiments across multiple benchmark datasets demonstrate that MIRRAMS consistently outperforms state-of-the-art methods under diverse missing-data scenarios—including MCAR, MAR, and MNAR—while also achieving new state-of-the-art performance on fully observed data. This dual efficacy underscores its strong generalization capability and stability across varying data completeness regimes.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multi-Robot Systems
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In real-world data analysis, missingness distributional shifts between training and test input datasets frequently occur, posing a significant challenge to achieving robust prediction performance. In this study, we propose a novel deep learning framework designed to address such shifts in missingness distributions. We begin by introducing a set of mutual information-based conditions, called MI robustness conditions, which guide a prediction model to extract label-relevant information while remaining invariant to diverse missingness patterns, thereby enhancing robustness to unseen missingness scenarios at test-time. To make these conditions practical, we propose simple yet effective techniques to derive loss terms corresponding to each and formulate a final objective function, termed MIRRAMS(Mutual Information Regularization for Robustness Against Missingness Shifts). As a by-product, our analysis provides a theoretical interpretation of the principles underlying consistency regularization-based semi-supervised learning methods, such as FixMatch. Extensive experiments across various benchmark datasets show that MIRRAMS consistently outperforms existing baselines and maintains stable performance across diverse missingness scenarios. Moreover, our approach achieves state-of-the-art performance even without missing data and can be naturally extended to address semi-supervised learning tasks, highlighting MIRRAMS as a powerful, off-the-shelf framework for general-purpose learning.

Problem

• Addressing robustness to missingness distribution shifts in training and test data
• Developing a deep learning framework invariant to diverse missingness patterns
• Enhancing prediction performance across unseen missingness scenarios

Innovation

• Mutual information conditions enhance robustness
• Loss terms derived for practical application
• State-of-the-art performance without missing data