Handling Incomplete Heterogeneous Data using a Data-Dependent Kernel

2025-01-08

AI Summary

Real-world heterogeneous (numerical, categorical, and mixed-type) data frequently exhibit non-random missingness—often not missing completely at random (MCAR)—posing significant challenges for similarity-based learning. To address this, we propose the Probability Mass Similarity Kernel (PMK), a novel, data-dependent kernel that requires no prior assumptions about the missingness mechanism or data type. PMK directly models feature-wise similarity from the observed empirical distribution, enabling unified treatment of heterogeneous features while implicitly capturing missingness patterns—thus eliminating the need for elaborate imputation or strong modeling assumptions. Evaluated across十余 diverse heterogeneous datasets with varying missingness mechanisms and rates, PMK consistently achieves statistically significant improvements over state-of-the-art methods in both classification and clustering tasks, demonstrating superior generalization and robustness to realistic missing-data scenarios.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Recommender Systems
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Handling incomplete data in real-world applications is a critical challenge due to two key limitations of existing methods: (i) they are primarily designed for numeric data and struggle with categorical or heterogeneous/mixed datasets; (ii) they assume that data is missing completely at random, which is often not the case in practice -- in reality, data is missing in patterns, leading to biased results if these patterns are not accounted for. To address these two limitations, this paper presents a novel approach to handling missing values using the Probability Mass Similarity Kernel (PMK), a data-dependent kernel, which does not make any assumptions about data types and missing mechanisms. It eliminates the need for prior knowledge or extensive pre-processing steps and instead leverages the distribution of observed data. Our method unifies the representation of diverse data types by capturing more meaningful pairwise similarities and enhancing downstream performance. We evaluated our approach across over 10 datasets with numerical-only, categorical-only, and mixed features under different missing mechanisms and rates. Across both classification and clustering tasks, our approach consistently outperformed existing techniques, demonstrating its robustness and effectiveness in managing incomplete heterogeneous data.

Problem

• Incomplete Data
• Mixed Data Analysis
• Missing Data Patterns

Innovation

• Probabilistic Mass Kernel
• Missing Data Handling
• Data Similarity Accuracy