Comparing representations of long clinical texts for the task of patient note-identification

2025-03-31

AI Summary

This study addresses patient-level precise matching of anonymized clinical notes, enabling duplicate record detection and patient similarity analysis. To tackle the challenges of modeling long clinical texts, we systematically evaluate multiple hierarchical and pretrained models—including HAN, HiTransformer, LongFormer, and BERT variants—and find that BERT-based models consistently outperform alternatives. We further propose and validate mean_max pooling—a novel aggregation strategy for deriving patient-level representations—which demonstrates superior effectiveness and strong cross-institutional generalizability. Evaluated on real-world data from MIMIC-IV and France’s Necker Hospital, BERT embeddings combined with mean_max pooling achieve state-of-the-art matching performance. This approach significantly enhances the robustness and practical utility of patient representations for lengthy clinical narratives. Our work establishes a reproducible, privacy-preserving methodology for clinical knowledge mining from anonymized text, advancing scalable and trustworthy secondary use of sensitive health data.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Representation Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In this paper, we address the challenge of patient-note identification, which involves accurately matching an anonymized clinical note to its corresponding patient, represented by a set of related notes. This task has broad applications, including duplicate records detection and patient similarity analysis, which require robust patient-level representations. We explore various embedding methods, including Hierarchical Attention Networks (HAN), three-level Hierarchical Transformer Networks (HTN), LongFormer, and advanced BERT-based models, focusing on their ability to process mediumto-long clinical texts effectively. Additionally, we evaluate different pooling strategies (mean, max, and mean_max) for aggregating wordlevel embeddings into patient-level representations and we examine the impact of sliding windows on model performance. Our results indicate that BERT-based embeddings outperform traditional and hierarchical models, particularly in processing lengthy clinical notes and capturing nuanced patient representations. Among the pooling strategies, mean_max pooling consistently yields the best results, highlighting its ability to capture critical features from clinical notes. Furthermore, the reproduction of our results on both MIMIC dataset and Necker hospital data warehouse illustrates the generalizability of these approaches to real-world applications, emphasizing the importance of both embedding methods and aggregation strategies in optimizing patient-note identification and enhancing patient-level modeling.

Problem

• Matching anonymized clinical notes to correct patients
• Evaluating embedding methods for clinical text processing
• Optimizing patient-level representations with pooling strategies

Innovation

• BERT-based embeddings outperform traditional models
• mean_max pooling captures critical clinical features
• sliding windows enhance lengthy note processing