Dynamic Feature Selection from Variable Feature Sets Using Features of Features

2025-03-12

AI Summary

This paper addresses dynamic feature selection (DFS) under the realistic setting where the set of measurable features varies across instances—challenging the conventional assumption of a fixed feature set. We formally define and solve the novel problem of “dynamic feature selection with variable-length measurable feature sets.” To tackle instance-level feature-set heterogeneity, we propose a “features-of-features” meta-representation framework: it employs meta-feature embeddings to encode varying feature availability patterns, integrates differentiable feature selection, and incorporates a cost-sensitive sequential decision mechanism. Evaluated on multiple benchmark datasets, our approach significantly reduces measurement cost while preserving predictive accuracy; notably, it improves cost-effectiveness (accuracy/cost) by 12.7% on average under highly volatile feature-set conditions. This work establishes a new paradigm for adaptive feature acquisition in resource-constrained environments.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Machine learning models usually assume that a set of feature values used to obtain an output is fixed in advance. However, in many real-world problems, a cost is associated with measuring these features. To address the issue of reducing measurement costs, various methods have been proposed to dynamically select which features to measure, but existing methods assume that the set of measurable features remains constant, which makes them unsuitable for cases where the set of measurable features varies from instance to instance. To overcome this limitation, we define a new problem setting for Dynamic Feature Selection (DFS) with variable feature sets and propose a deep learning method that utilizes prior information about each feature, referred to as ''features of features''. Experimental results on several datasets demonstrate that the proposed method effectively selects features based on the prior information, even when the set of measurable features changes from instance to instance.

Problem

• Dynamic feature selection with variable feature sets.
• Reducing measurement costs in real-world problems.
• Utilizing prior information about features for selection.

Innovation

• Dynamic Feature Selection with variable feature sets
• Utilizes prior information called 'features of features'
• Deep learning method adapts to changing measurable features