Data Imputation by Pursuing Better Classification: A Supervised Kernel-Based Method

2024-05-13

Pattern Recognition

AI Summary

To address the limitations of existing supervised imputation methods under high missingness rates (>60%)—namely, simplistic label usage, overly restrictive assumptions, and insufficient flexibility—this paper proposes a classification-performance-driven two-stage supervised kernel learning framework. In the first stage, perturbation-regularized collaborative learning is employed to construct a robust kernel matrix. In the second stage, this learned kernel matrix serves as a supervisory signal to guide block-coordinate-descent-based regression imputation. Crucially, the classification objective is deeply integrated into the imputation process, enabling joint optimization of the kernel matrix and the imputation model. Evaluated on four real-world datasets, the method consistently outperforms state-of-the-art approaches: under >60% missingness, it achieves an average 9.2% improvement in classification accuracy and a 31.5% reduction in imputation error.

Technology Category

• Machine Learning: Kernel Methods
• Game Theory and Economic Paradigms: Imperfect Information
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Data imputation, the process of filling in missing feature elements for incomplete data sets, plays a crucial role in data-driven learning. A fundamental belief is that data imputation is helpful for learning performance, and it follows that the pursuit of better classification can guide the data imputation process. While some works consider using label information to assist in this task, their simplistic utilization of labels lacks flexibility and may rely on strict assumptions. In this paper, we propose a new framework that effectively leverages supervision information to complete missing data in a manner conducive to classification. Specifically, this framework operates in two stages. Firstly, it leverages labels to supervise the optimization of similarity relationships among data, represented by the kernel matrix, with the goal of enhancing classification accuracy. To mitigate overfitting that may occur during this process, a perturbation variable is introduced to improve the robustness of the framework. Secondly, the learned kernel matrix serves as additional supervision information to guide data imputation through regression, utilizing the block coordinate descent method. The superiority of the proposed method is evaluated on four real-world data sets by comparing it with state-of-the-art imputation methods. Remarkably, our algorithm significantly outperforms other methods when the data is missing more than 60% of the features

Problem

• Supervised kernel method improves data imputation for classification
• Two-stage framework enhances similarity relationships using labels
• Robust imputation handles high missing data rates exceeding 60%

Innovation

• Supervised kernel optimization enhances classification accuracy
• Perturbation variable improves framework robustness
• Kernel-guided imputation via block coordinate descent