EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning

2026-04-12

AI Summary

This study addresses the tendency of current large language models (LLMs) to overfit to historical diagnoses in electronic health records (EHRs), which impairs their ability to detect emerging conditions requiring early intervention. To overcome this limitation, the authors propose EviCare, a novel framework that synergistically integrates deep learning models with LLMs. EviCare employs a deep model to generate candidate diagnoses and constructs adaptive context prompts by prioritizing set-based EHR evidence and relational diagnostic cues, thereby guiding the LLM toward evidence-driven, interpretable predictions. Experiments on the MIMIC-III and MIMIC-IV datasets demonstrate that EviCare improves average precision and accuracy by 20.65% and enhances performance on new-onset disease prediction by 30.97%, significantly outperforming existing approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in large language models (LLMs) have enabled promising progress in diagnosis prediction from electronic health records (EHRs). However, existing LLM-based approaches tend to overfit to historically observed diagnoses, often overlooking novel yet clinically important conditions that are critical for early intervention. To address this, we propose EviCare, an in-context reasoning framework that integrates deep model guidance into LLM-based diagnosis prediction. Rather than prompting LLMs directly with raw EHR inputs, EviCare performs (1) deep model inference for candidate selection, (2) evidential prioritization for set-based EHRs, and (3) relational evidence construction for novel diagnosis prediction. These signals are then composed into an adaptive in-context prompt to guide LLM reasoning in an accurate and interpretable manner. Extensive experiments on two real-world EHR benchmarks (MIMIC-III and MIMIC-IV) demonstrate that EviCare achieves significant performance gains, which consistently outperforms both LLM-only and deep model-only baselines by an average of 20.65\% across precision and accuracy metrics. The improvements are particularly notable in challenging novel diagnosis prediction, yielding average improvements of 30.97\%.

Problem

• diagnosis prediction
• large language models
• electronic health records
• novel diagnosis
• overfitting

Innovation

• in-context reasoning
• diagnosis prediction
• evidence prioritization
• deep model guidance
• electronic health records