Why Do Self-Harm Prediction Models Struggle to Generalise? Lexical and Semantic Variations in Emergency Department Triage Notes

2026-06-01

AI Summary

Self-harm prediction models often perform well within a single institution but suffer significant performance degradation when deployed across different hospitals, limiting their clinical utility. This study systematically investigates this limitation by comparing emergency department triage notes from two hospitals through natural language processing, term frequency analysis, feature importance assessment, and topic modeling. Despite substantial alignment in core thematic content, the analysis reveals marked differences in lexical choice and semantic expression between institutions, which critically undermine model generalizability. These findings provide empirical evidence that institutional linguistic variation is a key barrier to cross-site deployment and offer concrete directions for enhancing the robustness and transferability of self-harm detection models in real-world clinical settings.

Abstract

Self-harm presentations to emergency departments (EDs) are strongly associated with higher suicide risk. NLP models have shown robust performance in detecting self-harm from triage notes within single hospitals, yet performance often declines across institutions. To examine potential causes, we compare ED triage notes from two hospitals by analyzing lexical characteristics, highly associated predictive features, and salient topics. Our results reveal variation in lexical expression and feature importance related to self-harm across hospitals, despite consistent core themes such as self-poisoning and self-injury. These documentation differences are associated with reduced cross-site performance. Our findings provide insight into how institutional variation affects the identification of self-harm in clinical text and highlight potential methods to improve model generalisability.

Problem

• self-harm prediction
• model generalisation
• emergency department
• clinical text variation
• NLP

Innovation

• cross-institutional generalization
• lexical variation
• clinical NLP
• self-harm detection
• emergency department triage notes