Transferable Self-Harm Surveillance from Emergency Department Triage Notes Using an Evidence-Augmented Machine Learning Approach

2026-06-01

AI Summary

Current diagnostic code–based approaches for self-harm surveillance suffer from insufficient sensitivity, limiting their effectiveness in identifying self-harm cases in emergency departments. This study proposes a three-stage framework that integrates traditional machine learning with large language models to automatically screen for self-harm incidents and extract key supporting evidence from emergency triage notes, enabling fine-grained monitoring. The method achieves cross-institutional model transferability without site-specific fine-tuning, yielding AUPRC scores of 0.887 and 0.884 in internal and external validation, respectively. Prospective evaluation across three hospitals demonstrates robust performance, with the highest AUPRC reaching 0.881, and the system accurately identifies the primary method of self-harm with 95% precision.

Abstract

Self-harm is a major public health concern, but current surveillance relying on hospital presentations is inadequate due to the low sensitivity of diagnostic codes. Emergency Department (ED) triage notes, recorded at the initial point of contact, provide a succinct summary of presentations and an opportunity to identify self-harm. We developed a three-stage approach, augmenting traditional machine learning with large language model-based screening and evidence extraction to detect self-harm in ED triage notes. We assessed model transferability across three Australian hospitals. Our approach showed AUPRCs of 0.887 +/- 0.016 and 0.884 +/- 0.012 during internal and external validation. Prospectively, it achieved AUPRC of 0.881 +/- 0.008 at the development site, and 0.879 +/- 0.012 and 0.816 +/- 0.015 at two external sites without site-specific retraining. A key advantage of the approach is that it enables identification of the primary self-harm method with an accuracy of 95%, supporting more granular surveillance beyond binary classification.

Problem

• self-harm
• surveillance
• emergency department
• triage notes
• diagnostic codes

Innovation

• evidence-augmented machine learning
• large language model
• self-harm surveillance
• transferable model
• emergency department triage notes