Hypernetworks for Dynamic Feature Selection

2026-05-12

AI Summary

This work addresses the challenge of balancing the exponentially growing space of feature acquisition paths with model generalization in budget-constrained dynamic feature selection. To this end, we propose Hyper-DFS, the first approach to introduce hypernetworks into this task, which dynamically generates classifier parameters for any subset of features on demand. By integrating a Set Transformer, Hyper-DFS constructs a smooth conditional embedding space wherein semantically similar feature subsets are mapped to geometrically proximate regions. A masked embedding contrastive mechanism further reduces the upper bound of structural complexity, enabling strong zero-shot generalization to unseen feature subsets. Extensive experiments demonstrate that Hyper-DFS consistently outperforms existing methods on both synthetic and real-world tabular data and achieves comparable or superior performance on image datasets.

Abstract

Dynamic feature selection (DFS) is a machine learning framework in which features are acquired sequentially for individual samples under budget constraints. The exponential growth in the number of possible feature acquisition paths forces a DFS model to balance fitting specific scenarios against maintaining general performance, even when the feature space is moderate in size. In this paper, we study the structural limitations of existing DFS approaches to achieve an optimal solution. Then, we propose \textsc{Hyper-DFS}, a hypernetwork-based DFS approach that generates feature subset-specific classifier parameters on demand. We show that the use of hypernetworks compared to mask-embedding methods results in a smaller structural complexity bound. We also use a Set Transformer encoding to create a smooth conditioning space for the hypernetwork, so that functionally similar tasks are also geometrically close. In our benchmarks, \textsc{Hyper-DFS} outperforms all state-of-the-art approaches on synthetic and real-life tabular data. It is also competitive or superior across all image datasets tested, and shows substantially stronger zero-shot generalisation to feature subsets never seen during training than existing DFS approaches.

Problem

• Dynamic Feature Selection
• feature acquisition
• budget constraints
• generalization
• combinatorial complexity

Innovation

• hypernetwork
• dynamic feature selection
• Set Transformer
• zero-shot generalization
• structural complexity