Learning Accurate Models on Incomplete Data with Minimal Imputation

2025-03-18

AI Summary

Real-world data frequently contain missing values, and the conventional “impute-then-model” paradigm incurs substantial computational overhead while risking bias propagation. This paper proposes Minimal Imputation—a novel paradigm that formally defines the minimal imputation problem: identifying the smallest subset of missing entries to impute such that downstream model performance remains optimal. Grounded in statistical learning theory and combinatorial optimization, we develop both exact and efficient approximation algorithms applicable to linear regression, tree-based models, and other common learners. Extensive experiments demonstrate that our approach reduces imputation time and computational resource consumption by over 70% on average, while preserving or even improving predictive accuracy. By decoupling imputation necessity from completeness, Minimal Imputation fundamentally resolves the longstanding trade-off between imputation exhaustiveness and modeling fidelity.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Missing data often exists in real-world datasets, requiring significant time and effort for imputation to learn accurate machine learning (ML) models. In this paper, we demonstrate that imputing all missing values is not always necessary to achieve an accurate ML model. We introduce the concept of minimal data imputation, which ensures accurate ML models trained over the imputed dataset. Implementing minimal imputation guarantees both minimal imputation effort and optimal ML models. We propose algorithms to find exact and approximate minimal imputation for various ML models. Our extensive experiments indicate that our proposed algorithms significantly reduce the time and effort required for data imputation.

Problem

• Reduces time and effort for data imputation
• Ensures accurate ML models with minimal imputation
• Proposes algorithms for exact and approximate imputation

Innovation

• Minimal data imputation reduces ML training effort.
• Algorithms find exact and approximate minimal imputation.
• Ensures accurate ML models with less imputation time.