DeltaSHAP: Explaining Prediction Evolutions in Online Patient Monitoring with Shapley Values

2025-07-03

AI Summary

In clinical online monitoring, existing eXplainable AI (XAI) methods struggle to elucidate the dynamic drivers behind time-varying patient risk trajectories. To address this, we propose DeltaSHAP—the first Shapley-value-based attribution method explicitly designed for explaining *changes* in sequential predictions, rather than static risk scores. DeltaSHAP attributes the *difference* between consecutive predictions, capturing both the directionality (increase/decrease) and magnitude of each feature’s contribution to risk evolution. It constructs temporal attributions over realistic feature combinations and incorporates an efficient computation strategy to ensure real-time applicability. Additionally, we introduce a novel fidelity metric tailored to temporal attribution evaluation. On the MIMIC-III decompensation benchmark, DeltaSHAP achieves a 62% improvement in explanation quality and reduces inference latency by 33% compared to state-of-the-art XAI methods.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This study proposes DeltaSHAP, a novel explainable artificial intelligence (XAI) algorithm specifically designed for online patient monitoring systems. In clinical environments, discovering the causes driving patient risk evolution is critical for timely intervention, yet existing XAI methods fail to address the unique requirements of clinical time series explanation tasks. To this end, DeltaSHAP addresses three key clinical needs: explaining the changes in the consecutive predictions rather than isolated prediction scores, providing both magnitude and direction of feature attributions, and delivering these insights in real time. By adapting Shapley values to temporal settings, our approach accurately captures feature coalition effects. It further attributes prediction changes using only the actually observed feature combinations, making it efficient and practical for time-sensitive clinical applications. We also introduce new evaluation metrics to evaluate the faithfulness of the attributions for online time series, and demonstrate through experiments on online patient monitoring tasks that DeltaSHAP outperforms state-of-the-art XAI methods in both explanation quality as 62% and computational efficiency as 33% time reduction on the MIMIC-III decompensation benchmark. We release our code at https://github.com/AITRICS/DeltaSHAP.

Problem

• Explains changes in consecutive predictions for online patient monitoring
• Provides magnitude and direction of feature attributions in real time
• Adapts Shapley values for efficient clinical time series explanation

Innovation

• Adapts Shapley values for temporal clinical data
• Explains prediction changes with magnitude and direction
• Provides real-time feature attribution insights