🤖 AI Summary
Large language models (LLMs) frequently exhibit hallucinations due to ill-defined knowledge boundaries; existing instruction tuning approaches overemphasize answer generation while neglecting the explicit recognition and articulation of “unknown” queries. To address this, we propose Uncertainty-Aware Two-stage Instruction Tuning (US-Tuning), the first method to integrate causal prompting into knowledge boundary awareness training, thereby decoupling knowledge boundary identification from instruction following. Our approach comprises three components: (1) constructing knowledge boundary-annotated data via context-based question answering, (2) causal prompting engineering to elicit uncertainty-aware responses, and (3) an uncertainty-aware loss function that jointly optimizes boundary detection and response fidelity. Experiments on Llama2-7B demonstrate a 34.7% improvement in unknown-query identification accuracy, outperforming GPT-4 by 4.2% on overall task performance, while significantly reducing hallucinations and enhancing parametric knowledge faithfulness.
📝 Abstract
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context. While instructing LLMs to acknowledge knowledge limitations by responding with"I don't know"appears promising, we find that models consistently struggle with admitting knowledge gaps. This challenge may originate from current instruction datasets that emphasise answer generation over knowledge boundary awareness. To address this limitation, we introduce Uncertainty-and-Sensitivity-Aware Tuning (US-Tuning), a novel two-stage approach for contextual question answering (QA). The first stage enhances LLMs' ability to recognise their knowledge boundaries, while the second stage reinforces instruction adherence through carefully designed causal prompts. Our experimental results demonstrate that US-Tuning not only significantly reduces incorrect answers in contextual QA but also improves models' faithfulness to their parametric knowledge, mitigating hallucinations in general QA tasks. Our fine-tuned Llama2-7B model achieves up to a 34.7% improvement in handling out-of-knowledge questions and outperforms GPT-4 by 4.2% in overall performance.