Stochastic Encodings for Active Feature Acquisition

2025-08-03

AI Summary

Active Feature Acquisition (AFA) aims to dynamically select the most informative unobserved features for each test instance to improve predictive performance. Existing approaches either rely on hard-to-train reinforcement learning or employ myopic greedy criteria based on mutual information. This paper proposes a supervised latent-variable modeling framework: it performs probabilistic inference over multiple plausible realizations of unobserved features via stochastic encoding in a latent space, jointly capturing their complex dependencies with the target label. This enables non-greedy, globally optimal feature selection. By avoiding reinforcement learning altogether, the method eliminates training instability while overcoming the local optimality limitation inherent in greedy strategies. Extensive experiments on multiple synthetic and real-world datasets demonstrate that our approach consistently and significantly outperforms diverse state-of-the-art baselines.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Active Feature Acquisition is an instance-wise, sequential decision making problem. The aim is to dynamically select which feature to measure based on current observations, independently for each test instance. Common approaches either use Reinforcement Learning, which experiences training difficulties, or greedily maximize the conditional mutual information of the label and unobserved features, which makes myopic acquisitions. To address these shortcomings, we introduce a latent variable model, trained in a supervised manner. Acquisitions are made by reasoning about the features across many possible unobserved realizations in a stochastic latent space. Extensive evaluation on a large range of synthetic and real datasets demonstrates that our approach reliably outperforms a diverse set of baselines.

Problem

• Dynamic feature selection for sequential decision making
• Overcoming training difficulties in Reinforcement Learning approaches
• Avoiding myopic acquisitions in mutual information methods

Innovation

• Uses latent variable model for feature acquisition
• Trains model in supervised manner
• Reasons across stochastic latent space