No Black Boxes: Interpretable and Interactable Predictive Healthcare with Knowledge-Enhanced Agentic Causal Discovery

2025-05-22

AI Summary

To address the clinical challenge of limited interpretability and interactivity in EHR-based deep learning models, this work proposes a knowledge-enhanced, agent-driven causal discovery framework. Methodologically, it introduces the first causal reasoning architecture integrating a personalized medical knowledge base with an embodied large language model (LLM), combining knowledge graph embedding, constraint-based (PC) and continuous optimization (NOTEARS) causal discovery, EHR temporal modeling, and interactive prompt engineering—enabling explicit prediction attribution, dynamic knowledge injection, and human-AI collaborative decision-making. On MIMIC-III/IV, diagnostic prediction achieves AUC > 0.92. Case studies demonstrate that clinicians can accurately comprehend, trace, and refine model reasoning paths, with mean interactive response latency < 1.2 seconds. The core contribution is the establishment of the first clinical decision-support agent paradigm supporting real-time causal explanation and bidirectional knowledge interaction.

Technology Category

• Machine Learning: Causal Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Deep learning models trained on extensive Electronic Health Records (EHR) data have achieved high accuracy in diagnosis prediction, offering the potential to assist clinicians in decision-making and treatment planning. However, these models lack two crucial features that clinicians highly value: interpretability and interactivity. The ``black-box'' nature of these models makes it difficult for clinicians to understand the reasoning behind predictions, limiting their ability to make informed decisions. Additionally, the absence of interactive mechanisms prevents clinicians from incorporating their own knowledge and experience into the decision-making process. To address these limitations, we propose II-KEA, a knowledge-enhanced agent-driven causal discovery framework that integrates personalized knowledge databases and agentic LLMs. II-KEA enhances interpretability through explicit reasoning and causal analysis, while also improving interactivity by allowing clinicians to inject their knowledge and experience through customized knowledge bases and prompts. II-KEA is evaluated on both MIMIC-III and MIMIC-IV, demonstrating superior performance along with enhanced interpretability and interactivity, as evidenced by its strong results from extensive case studies.

Problem

• Enhances interpretability of predictive healthcare models
• Improves interactivity for clinician decision-making
• Integrates personalized knowledge with causal discovery

Innovation

• Knowledge-enhanced agent-driven causal discovery framework
• Integrates personalized knowledge databases and agentic LLMs
• Enhances interpretability and interactivity through explicit reasoning