From Policy to Logic for Efficient and Interpretable Coverage Assessment

2026-01-03

arXiv.org

AI Summary

This work addresses the susceptibility of large language models (LLMs) to hallucination and inconsistency when interpreting complex healthcare coverage policies, which undermines decision reliability. To mitigate this, the authors propose a hybrid approach that integrates a coverage-aware retriever with symbolic rule-based reasoning to transform policy texts into explicit facts and rules, thereby generating auditable reasoning chains. By grounding LLM inferences in structured, verifiable logic, the method substantially reduces the number of LLM invocations, achieving a 44% reduction in inference cost while simultaneously improving F1 score by 4.5%. This framework effectively balances computational efficiency, accuracy, and traceability, offering a more trustworthy solution for policy interpretation in high-stakes healthcare settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in interpreting lengthy, complex legal and policy language. However, their reliability can be undermined by hallucinations and inconsistencies, particularly when analyzing subjective and nuanced documents. These challenges are especially critical in medical coverage policy review, where human experts must be able to rely on accurate information. In this paper, we present an approach designed to support human reviewers by making policy interpretation more efficient and interpretable. We introduce a methodology that pairs a coverage-aware retriever with symbolic rule-based reasoning to surface relevant policy language, organize it into explicit facts and rules, and generate auditable rationales. This hybrid system minimizes the number of LLM inferences required which reduces overall model cost. Notably, our approach achieves a 44% reduction in inference cost alongside a 4.5% improvement in F1 score, demonstrating both efficiency and effectiveness.

Problem

• coverage assessment
• large language models
• policy interpretation
• hallucination
• medical policy review

Innovation

• coverage-aware retrieval
• symbolic reasoning
• interpretable AI
• LLM efficiency
• policy interpretation