Evaluation of the impact of expert knowledge: How decision support scores impact the effectiveness of automatic knowledge-driven feature engineering (aKDFE)

2025-04-08

AI Summary

This study addresses adverse drug event (ADE) prediction from electronic health records (EHRs). We propose automated Knowledge-Driven Feature Engineering (aKDFE), comprising two stages: event-driven feature generation and patient-centered temporal transformation. The core innovation lies in the patient-centered transformation, which maps sequential clinical events onto the patient level and aggregates longitudinal historical information—substantially enhancing model discriminability (achieving high AUROC). To our knowledge, this is the first systematic evaluation of domain knowledge integration for ADE prediction. We find that expert risk scores and administration-route features from Janusmed Riskprofile CDSS yield no statistically significant improvement, whereas prior medical history features demonstrate strong predictive power. Our results indicate that patient-centered modeling constitutes a critical breakthrough, and external knowledge must be carefully aligned with EHR’s structured representation paradigm to meaningfully augment ADE prediction performance.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Knowledge Representation and Reasoning: Knowledge Engineering
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Adverse Drug Events (ADEs), harmful medication effects, pose significant healthcare challenges, impacting patient safety and costs. This study evaluates automatic Knowledge-Driven Feature Engineering (aKDFE) for improved ADE prediction from Electronic Health Record (EHR) data, comparing it with automated event-based Knowledge Discovery in Databases (KDD). We investigated how incorporating domain-specific ADE risk scores for prolonged heart QT interval, extracted from the Janusmed Riskprofile (Janusmed) Clinical Decision Support System (CDSS), affects prediction performance using EHR data and medication handling events. Results indicate that, while aKDFE step 1 (event-based feature generation) alone did not significantly improve ADE prediction performance, aKDFE step 2 (patient-centric transformation) enhances the prediction performance. High Area Under the Receiver Operating Characteristic curve (AUROC) values suggest strong feature correlations to the outcome, aligning with the predictive power of patients' prior healthcare history for ADEs. Statistical analysis did not confirm that incorporating the Janusmed information (i) risk scores and (ii) medication route of administration into the model's feature set enhanced predictive performance. However, the patient-centric transformation applied by aKDFE proved to be a highly effective feature engineering approach. Limitations include a single-project focus, potential bias from machine learning pipeline methods, and reliance on AUROC. In conclusion, aKDFE, particularly with patient-centric transformation, improves ADE prediction from EHR data. Future work will explore attention-based models, event feature sequences, and automatic methods for incorporating domain knowledge into the aKDFE framework.

Problem

• Evaluates aKDFE for better ADE prediction from EHR data
• Assesses impact of expert risk scores on prediction performance
• Compares patient-centric transformation with event-based feature generation

Innovation

• Automatic Knowledge-Driven Feature Engineering (aKDFE)
• Patient-centric transformation enhances prediction
• Incorporates domain-specific ADE risk scores