X-FEMR: A Token-level Explainable Approach for Electronic Health Records Foundation Models using Transformer-based Models

2026-07-07

AI Summary

This work addresses the limited interpretability of foundation models for electronic health records (FEMRs), which undermines clinical trust and raises concerns about potential biases. To bridge this gap, the authors propose the first token-level interpretability method tailored for FEMRs: a Transformer-based surrogate model trained on input–output pairs from the original FEMR to faithfully replicate its predictions while preserving temporal dynamics and identifying salient predictive features. The study further introduces a clinical alignment metric to evaluate the consistency between generated explanations and established medical knowledge. Experimental results demonstrate that the surrogate model accurately approximates the original FEMR’s outputs, and its token-level attributions exhibit strong agreement with clinically validated key features, thereby substantially enhancing model credibility in healthcare settings.

Abstract

Foundation Models for Electronic Health Records (FEMRs) are pretrained on large-scale structured patient data, enabling them to convert longitudinal patient trajectories into generalizable representations for diverse clinical prediction tasks. Despite their effectiveness, FEMRs remain black-box models, raising concerns about bias, interpretability, and clinical trust. To address this, we propose the first token-level explainability approach for FEMRs. We train a Transformer-based surrogate model on input-output pairs from the FEMR across two prediction tasks, approximating its behavior while preserving temporal dynamics. We identify the most influential tokens, providing insights into how FEMRs leverage different aspects of patient history for predictions. To evaluate clinical relevance, we introduce a novel clinical alignment metric that quantifies the correspondence between the surrogate model's key tokens and clinically validated features. Our results demonstrate that the surrogate closely approximates FEMR predictions and that token-level explanations align well with clinical knowledge, offering a practical framework for interpretable and trustworthy clinical AI.

Problem

• Foundation Models
• Electronic Health Records
• Explainability
• Interpretability
• Clinical Trust

Innovation

• token-level explainability
• Foundation Models for Electronic Health Records
• Transformer-based surrogate model
• clinical alignment metric
• interpretable AI