Interpretable Uncertainty for Adaptive Retrieval and Reasoning in Question Answering

2026-07-08

AI Summary

Large language models (LLMs) in question answering are often hindered by hallucinations and a lack of transparency. Existing retrieval-augmented approaches either rely on opaque strategies or require inefficient multi-step prompting. This work proposes an uncertainty-aware framework grounded in the model’s internal representations, which, within a single forward pass, decomposes uncertainty into two interpretable signals: “knowledge insufficiency” and “knowledge ambiguity or conflict.” These signals are used to adaptively trigger external retrieval or additional reasoning steps. By leveraging hidden states to efficiently estimate uncertainty, the method enables transparent and dynamic control over when to retrieve or reason further, offering a novel pathway toward building more interpretable and efficient question-answering systems.

Abstract

Large language models (LLMs) achieve a strong performance in question answering (QA), but remain prone to hallucinations and suffer from limited transparency. Retrieval-augmented generation (RAG) can improve factuality, yet decisions about when and how to retrieve from external resources are typically based on opaque policies or computationally inefficient multi-step prompting procedures. We propose an uncertainty-aware framework for adaptive QA based on explicit signals derived from LLM internal representations. We distinguish between knowledge insufficiency and knowledge ambiguity or conflict, and efficiently estimate these from hidden states in a single forward pass. These signals guide system behaviour: RAG is triggered when knowledge is insufficient, while additional reasoning is applied when ambiguity or conflict is high. By grounding adaptive decisions in decomposed and efficiently estimable uncertainty signals, this approach provides a transparent and practical alternative to existing retrieval and reasoning strategies supporting the design of interpretable user-facing tools.

Problem

• uncertainty
• retrieval-augmented generation
• interpretability
• question answering
• hallucination

Innovation

• uncertainty-aware
• adaptive retrieval
• retrieval-augmented generation
• interpretable AI
• language model internal representations