Prediction Models That Learn to Avoid Missing Values

2025-05-06

AI Summary

Missing values during inference degrade model accuracy and impair interpretability. Method: This paper proposes the “Missingness-Avoidance” (MA) paradigm, which proactively suppresses model reliance on missing features during training. Contribution/Results: We introduce the first context-aware regularization mechanisms to inhibit missing-feature dependency—specifically designed for decision trees, tree ensembles (e.g., random forests and gradient-boosted trees), and sparse linear models (e.g., LASSO). These mechanisms explicitly constrain the frequency of missing-feature invocation via task-specific objective functions. Evaluated on multiple real-world datasets, MA reduces missing-feature usage by an average of 37% while maintaining predictive accuracy comparable to state-of-the-art baselines. Crucially, MA preserves model transparency and traceable decision paths, thereby achieving a favorable trade-off between high accuracy and strong interpretability.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Graphical Models
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Handling missing values at test time is challenging for machine learning models, especially when aiming for both high accuracy and interpretability. Established approaches often add bias through imputation or excessive model complexity via missingness indicators. Moreover, either method can obscure interpretability, making it harder to understand how the model utilizes the observed variables in predictions. We propose missingness-avoiding (MA) machine learning, a general framework for training models to rarely require the values of missing (or imputed) features at test time. We create tailored MA learning algorithms for decision trees, tree ensembles, and sparse linear models by incorporating classifier-specific regularization terms in their learning objectives. The tree-based models leverage contextual missingness by reducing reliance on missing values based on the observed context. Experiments on real-world datasets demonstrate that MA-DT, MA-LASSO, MA-RF, and MA-GBT effectively reduce the reliance on features with missing values while maintaining predictive performance competitive with their unregularized counterparts. This shows that our framework gives practitioners a powerful tool to maintain interpretability in predictions with test-time missing values.

Problem

• Handling missing values without bias or complexity
• Maintaining model interpretability with missing data
• Reducing reliance on missing features during prediction

Innovation

• Missingness-avoiding (MA) machine learning framework
• Classifier-specific regularization for decision trees
• Contextual missingness reduction in tree-based models