Zero-Shot Active Feature Acquisition via LLM-Elicitation

2026-06-17

AI Summary

This work addresses the challenge of active feature acquisition in label-scarce settings—such as clinical diagnosis—where conventional methods rely heavily on abundant annotated data. The authors propose a novel approach that leverages large language models (LLMs) as unsupervised sources of domain knowledge, using structured prompts to extract unary biases and pairwise co-variation statistics required for Markov random fields. To mitigate issues of missing discriminative information and ambiguous normalization, they introduce a maximum-entropy closure strategy. By decoupling knowledge provision from decision planning, their method enables zero-shot active feature acquisition. Evaluated on an inflammatory bowel disease cohort, it significantly outperforms existing approaches, particularly excelling in top-k identification of the most diagnostically challenging cases—surpassing both the LLM’s own belief scores and baseline performance under ground-truth labels.

Abstract

Active feature acquisition (AFA) sequentially selects which features to observe to reach a classification or ranking decision. Its central limitation is reliance on large amount of labeled data to fit probabilistic models guiding acquisition. Large language models (LLMs) supply unsupervised domain knowledge, but are poor sequential planners. Asking one to both know and decide conflates capabilities best kept separate. Here, we develop a framework for zero-shot AFA through disciplined elicitation: asking the LLM only for what it can be trusted to return, the unary deviations and pairwise co-variations that are the sufficient statistics of a Markov random field (MRF). We apply our framework to two settings: binary classification and top-$k$ identification. In practice, the LLM reliably returns only discriminative statistics, what distinguishes the classes rather than each class in isolation, which precludes classical AFA. We apply a maximum-entropy closure that resolves this gauge ambiguity. We evaluate on a cohort of Inflammatory Bowel Disease (IBD) patients, an active clinical setting where diagnostic ambiguity and patient heterogeneity obstruct stable treatment strategies. Our framework outperforms the LLM both on real labels and on its own extracted beliefs. Where it matters most, on the hardest patients, our top-$k$ acquisition policy markedly outperforms all existing methods.

Problem

• Zero-Shot Active Feature Acquisition
• Large Language Models
• Markov Random Field
• Feature Selection
• Sequential Decision Making

Innovation

• Zero-Shot Active Feature Acquisition
• LLM Elicitation
• Markov Random Field
• Maximum-Entropy Closure
• Discriminative Statistics