Dynamic Feature Selection based on Rule-based Learning for Explainable Classification with Uncertainty Quantification

2025-08-04

AI Summary

To address the lack of interpretability and uncertainty quantification in dynamic feature selection (DFS), this paper proposes a rule-learning-based interpretable DFS framework. The method performs sample-level adaptive feature selection guided by conditional mutual information (CMI), enhances computational efficiency via constrained rule search space, and introduces an explicit uncertainty quantification module that models confidence for each feature query. Crucially, it establishes, for the first time, the equivalence between the CMI-based greedy strategy and global prediction consistency. Evaluated on multiple benchmark datasets, the proposed approach achieves performance comparable to state-of-the-art greedy and reinforcement learning–based DFS methods, while significantly improving decision transparency, model interpretability, and inference efficiency—making it particularly suitable for high-stakes, trust-critical applications such as clinical decision support.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Rule Mining & Pattern Mining
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Dynamic feature selection (DFS) offers a compelling alternative to traditional, static feature selection by adapting the selected features to each individual sample. Unlike classical methods that apply a uniform feature set, DFS customizes feature selection per sample, providing insight into the decision-making process for each case. DFS is especially significant in settings where decision transparency is key, i.e., clinical decisions; however, existing methods use opaque models, which hinder their applicability in real-life scenarios. This paper introduces a novel approach leveraging a rule-based system as a base classifier for the DFS process, which enhances decision interpretability compared to neural estimators. We also show how this method provides a quantitative measure of uncertainty for each feature query and can make the feature selection process computationally lighter by constraining the feature search space. We also discuss when greedy selection of conditional mutual information is equivalent to selecting features that minimize the difference with respect to the global model predictions. Finally, we demonstrate the competitive performance of our rule-based DFS approach against established and state-of-the-art greedy and RL methods, which are mostly considered opaque, compared to our explainable rule-based system.

Problem

• Dynamic feature selection adapts features per sample for explainable classification
• Existing opaque DFS models lack transparency for critical applications like clinical decisions
• Proposes rule-based DFS with uncertainty quantification and computational efficiency

Innovation

• Rule-based system enhances interpretability in feature selection
• Quantitative uncertainty measure for each feature query
• Constrained feature search space reduces computational load