Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs

2026-07-03

AI Summary

This work addresses the suboptimal performance of large language models (LLMs) in multi-turn active information-gathering tasks, often due to inefficient questioning strategies. The authors propose ASIG, a novel approach that, for the first time, amortizes Bayesian experimental design (BED) within LLMs by leveraging expected information gain as a reward signal and fine-tuning the model with grouped relative policy optimization. ASIG substantially enhances the model’s active reasoning capabilities: on the 20 Questions task, a 7B-parameter model achieves more than a two-fold increase in success rate while reducing inference cost by over 25×. Furthermore, the method demonstrates strong generalization on the unseen MediQ medical diagnosis benchmark, establishing a highly efficient and transferable framework for multi-turn information acquisition.

Abstract

Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings. We introduce Amortised Sequential Information Gathering (ASIG), a fine-tuning approach that amortises Bayesian Experimental Design (BED) into LLM policies via a multi-turn extension of Group Relative Policy Optimisation with an Expected Information Gain reward. Evaluated on the 20 Questions task, ASIG more than doubles the success rate of the 7B base model and reduces inference cost by over $25\times$ relative to BED-LLM, a competitive inference-time baseline. Applied to MediQ, a medical diagnosis benchmark unseen during training, ASIG improves information-seeking performance at the 7B scale, suggesting that the learned strategies can transfer out of distribution. Our findings show that amortising BED into LLM policies provides an effective and computationally efficient approach to sequential information gathering.

Problem

• Sequential Information Gathering
• Large Language Models
• Bayesian Experimental Design
• Multi-turn Interaction
• Information Seeking

Innovation

• Amortised Sequential Information Gathering
• Bayesian Experimental Design
• Expected Information Gain
• Large Language Models
• Multi-turn Decision Making