Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

2025-01-28

AI Summary

To address the challenge of low diagnostic prediction accuracy arising from limited patient information and a large number of disease categories in clinical settings, this paper proposes MERA—a novel framework for medical diagnosis. First, it introduces a hierarchical contrastive learning mechanism to improve ranking quality over candidate disease sets. Second, it incorporates a medical concept memory fine-tuning module that explicitly bridges unstructured clinical narratives with structured ICD codes. Third, it enhances model generalizability via large language model (LLM) adaptation and code–text alignment strategies. Evaluated on MIMIC-III and MIMIC-IV, MERA achieves state-of-the-art performance in diagnosis prediction, significantly outperforming existing methods. Moreover, it improves clinical interpretability of predictions and advances the practical applicability of generative LMs in real-world healthcare scenarios.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Clinical diagnosis prediction models, when provided with a patient's medical history, aim to detect potential diseases early, facilitating timely intervention and improving prognostic outcomes. However, the inherent scarcity of patient data and large disease candidate space often pose challenges in developing satisfactory models for this intricate task. The exploration of leveraging Large Language Models (LLMs) for encapsulating clinical decision processes has been limited. We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes. Experimental results on MIMIC-III and IV datasets show that MERA achieves the state-of-the-art diagnosis prediction performance and dramatically elevates the diagnosis prediction capabilities of generative LMs.

Problem

• Clinical Diagnostics
• Language Models
• Prediction Accuracy

Innovation

• MERA
• Clinical Diagnosis Prediction
• Large Language Model Integration