Search papers, labs, and topics across Lattice.
This paper introduces a novel framework for zero-shot active feature acquisition (AFA) that leverages large language models (LLMs) to elicit only the necessary statistics for classification and ranking tasks, specifically focusing on unary deviations and pairwise co-variations. By separating the knowledge retrieval from decision-making, the approach effectively circumvents the limitations of traditional AFA that require extensive labeled data. In practical applications, particularly in diagnosing Inflammatory Bowel Disease (IBD), the proposed method significantly outperforms existing techniques, especially in challenging cases, demonstrating its potential for real-world clinical settings.
Eliciting only the essential statistics from LLMs can dramatically enhance feature acquisition in complex clinical scenarios, outperforming traditional methods even in the toughest cases.
Active feature acquisition (AFA) sequentially selects which features to observe to reach a classification or ranking decision. Its central limitation is reliance on large amount of labeled data to fit probabilistic models guiding acquisition. Large language models (LLMs) supply unsupervised domain knowledge, but are poor sequential planners. Asking one to both know and decide conflates capabilities best kept separate. Here, we develop a framework for zero-shot AFA through disciplined elicitation: asking the LLM only for what it can be trusted to return, the unary deviations and pairwise co-variations that are the sufficient statistics of a Markov random field (MRF). We apply our framework to two settings: binary classification and top-$k$ identification. In practice, the LLM reliably returns only discriminative statistics, what distinguishes the classes rather than each class in isolation, which precludes classical AFA. We apply a maximum-entropy closure that resolves this gauge ambiguity. We evaluate on a cohort of Inflammatory Bowel Disease (IBD) patients, an active clinical setting where diagnostic ambiguity and patient heterogeneity obstruct stable treatment strategies. Our framework outperforms the LLM both on real labels and on its own extracted beliefs. Where it matters most, on the hardest patients, our top-$k$ acquisition policy markedly outperforms all existing methods.
