🤖 AI Summary
This work addresses the limitations of existing post-hoc explainable AI (XAI) methods in electrocardiogram (ECG) classification, which often produce explanations misaligned with clinically relevant features, obscuring whether models rely on genuine pathological signals. The authors propose the first guideline-based global evaluation framework, leveraging cardiologist-defined critical ECG regions from clinical guidelines as ground truth to systematically assess the reliability of 13 gradient-based XAI methods across four binary classification models on the PTB-XL dataset. By aggregating multi-beat explanations and employing Spearman correlation alongside region-wise attribution ratios, they reveal that most methods disproportionately emphasize high-amplitude QRS complexes while neglecting diagnostically crucial low-amplitude ST segments—for instance, LRP-ε assigns only 4.6% attribution to ischemic ST segments compared to LRP-SIGN’s 63.8%. Alarmingly, nine out of thirteen methods perform below random baseline for at least one pathology, exposing a systemic failure of vision-domain XAI techniques when naively transferred to medical time-series signals.
📝 Abstract
Explainable AI (XAI) is used to assess whether artificial intelligence models rely on meaningful patterns, yet explanations that appear plausible for individual predictions may systematically misrepresent model behavior. This is particularly problematic in medicine, where models may rely on irrelevant signal characteristics rather than disease-specific patterns without being recognizable. We address this challenge using electrocardiogram (ECG) data, for which clinical guidelines provide explicit knowledge about diagnostically relevant signal regions. We introduce a global, guideline-grounded framework that aggregates explanations across heartbeats to evaluate them against clinically defined regions of interest. Using four binary classifiers trained on PTB-XL, we assess 13 gradient-based methods across two categories of patterns: low-amplitude segments and high-amplitude QRS morphology. Our results reveal a systematic failure of methods transferred from computer vision. Their explanations often follow signal amplitude rather than clinical relevance, with mean Spearman correlations up to 0.69, leading them to overlook diagnostically decisive low-amplitude regions. For ischemia, LRP-$ε$ assigns only 4.6% of relevance to the ST segment, compared with 63.8% for LRP-SIGN. Nine of 13 methods fall below chance for at least one condition, indicating inconsistent reliability across patterns. These findings show that global, domain-grounded evaluation can uncover systematic explanation failures not obvious from sample-level heatmaps.