Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling

2025-03-04

AI Summary

Large language models (LLMs) frequently generate hallucinations beyond their knowledge boundaries, while existing uncertainty estimation or query rejection methods suffer from either low efficiency or compromised task utility. To address this, we propose the Explicit Knowledge Boundary Modeling (EKBM) framework—the first to explicitly model knowledge boundaries by coupling dual cognitive systems: “fast thinking” (efficient response generation) and “slow thinking” (deliberate refinement). EKBM introduces three core components: confidence-annotated data generation, a goal-directed slow-thinking refinement module, and an uncertainty-aware hybrid supervised training mechanism—jointly optimizing self-awareness alignment and task performance. Evaluated on dialogue state tracking, EKBM substantially outperforms uncertainty-based baselines: refinement-stage accuracy improves significantly, while computational overhead increases by less than 15%. Thus, EKBM achieves a favorable trade-off between reliability and practicality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) frequently hallucinate due to misaligned self-awareness, generating erroneous outputs when addressing queries beyond their knowledge boundaries. While existing approaches mitigate hallucinations via uncertainty estimation or query rejection, they suffer from computational inefficiency or sacrificed helpfulness. To address these issues, we propose the Explicit Knowledge Boundary Modeling (EKBM) framework, integrating fast and slow reasoning systems to harmonize reliability and usability. The framework first employs a fast-thinking model to generate confidence-labeled responses, enabling immediate use of high-confidence outputs. For uncertain predictions, a slow refinement model conducts targeted reasoning to improve accuracy. To align model behavior with our proposed object, we propose a hybrid training pipeline, enhancing self-awareness without degrading task performance. Evaluations on dialogue state tracking tasks demonstrate that EKBM achieves superior model reliability over uncertainty-based baselines. Further analysis reveals that refinement substantially boosts accuracy while maintaining low computational overhead. Our work establishes a scalable paradigm for advancing LLM reliability and balancing accuracy and practical utility in error-sensitive applications.

Problem

• Mitigate LLM hallucinations via knowledge boundary modeling
• Balance reliability and usability in LLM responses
• Enhance accuracy with low computational overhead

Innovation

• Integrates fast and slow reasoning systems
• Employs confidence-labeled responses for reliability
• Uses hybrid training to enhance self-awareness