LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering

2025-09-13

AI Summary

Large language models (LLMs) suffer from hallucination and struggle to integrate complex domain-expert mental models—particularly under training data scarcity—thereby compromising decision-support reliability. Method: We propose a causal prompt optimization framework grounded in Expert Mental Models (EMM). First, we introduce a novel four-step EMM algorithm that pioneers the use of monotonic Boolean functions and multi-valued logic for computationally tractable and scalable mental-structure modeling. Second, we integrate retrieval-augmented generation, causal prompt engineering, human-in-the-loop learning, and factor-wise hierarchical modeling to realize a generalized, EMM-driven personalized prompt generation mechanism. Results: Evaluated on proposal-response decision-making tasks, our framework significantly reduces hallucination rates while improving reasoning consistency and decision-support accuracy—demonstrating both empirical effectiveness and practical applicability in expert-augmented AI systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Difficult decision-making problems abound in various disciplines and domains. The proliferation of generative techniques, especially large language models (LLMs), has excited interest in using them for decision support. However, LLMs cannot yet resolve missingness in their training data, leading to hallucinations. Retrieval-Augmented Generation (RAG) enhances LLMs by incorporating external information retrieval, reducing hallucinations and improving accuracy. Yet, RAG and related methods are only partial solutions, as they may lack access to all necessary sources or key missing information. Even everyday issues often challenge LLMs' abilities. Submitting longer prompts with context and examples is one approach to address knowledge gaps, but designing effective prompts is non-trivial and may not capture complex mental models of domain experts. For tasks with missing critical information, LLMs are insufficient, as are many existing systems poorly represented in available documents. This paper explores how LLMs can make decision-making more efficient, using a running example of evaluating whether to respond to a call for proposals. We propose a technology based on optimized human-machine dialogue and monotone Boolean and k-valued functions to discover a computationally tractable personal expert mental model (EMM) of decision-making. Our EMM algorithm for LLM prompt engineering has four steps: (1) factor identification, (2) hierarchical structuring of factors, (3) generating a generalized expert mental model specification, and (4) generating a detailed generalized expert mental model from that specification.

Problem

• Reducing LLM hallucinations in decision-making via expert mental models
• Addressing missing information in LLMs through causal prompt engineering
• Developing computational methods to capture domain expert decision processes

Innovation

• Causal prompt engineering reduces hallucinations
• Expert mental model algorithm structures decisions
• Monotone Boolean functions optimize human-machine dialogue