Towards Truly Unsupervised Evaluation of Feature Selection

2026-08-12

AI Summary

This work addresses a critical flaw in existing unsupervised feature selection evaluation methods, which implicitly rely on label information and thereby violate the fundamental principle of unsupervised learning. To resolve this issue, the paper proposes the first genuinely unsupervised evaluation framework that operates entirely without access to labels. The framework jointly leverages principal component analysis (PCA) and optimal transport theory to assess the quality of feature subsets. It not only demonstrates that prevailing evaluation approaches are essentially supervised in nature but also provides a theoretically sound and practically effective alternative for unsupervised feature selection. Extensive experiments validate the framework’s validity and superiority, establishing its potential as a principled benchmark for future research in unsupervised feature selection.

Abstract

Feature selection is one of the most important and fundamental tasks in data mining, tackled by a family of methods with an established set of evaluation techniques to measure the quality of a specific method. Most of the methods commonly used for the unsupervised evaluation of feature selection algorithms suffer from critical design flaws which question their unsupervised nature. In this paper, we provide a critical discussion on the established allegedly unsupervised evaluation techniques, and shed light on the reasons why they are not truly unsupervised but, at best, supervised evaluation under an unsupervised downstream task. We also propose a novel, truly unsupervised evaluation framework to measure the quality of the feature selection algorithms without any form of information about the labels. The proposed framework utilizes unsupervised Principal Component Analysis, and optimal transport to measure the quality of the feature selection methods in a truly unsupervised manner.

Problem

• unsupervised evaluation
• feature selection
• label information
• evaluation framework

Innovation

• unsupervised evaluation
• feature selection
• principal component analysis
• optimal transport
• truly unsupervised