Search papers, labs, and topics across Lattice.
This paper introduces a token-level explainability approach for Foundation Models in Electronic Health Records (FEMRs) by training a Transformer-based surrogate model that captures input-output dynamics across clinical prediction tasks. By identifying influential tokens, the method reveals how FEMRs utilize patient history for predictions, addressing concerns about interpretability and bias in clinical AI. The evaluation shows that the surrogate model's predictions closely align with FEMR outputs and clinically validated features, enhancing trust in AI-driven healthcare solutions.
Token-level explanations reveal how FEMRs leverage patient history, bridging the gap between black-box models and clinical trust.
Foundation Models for Electronic Health Records (FEMRs) are pretrained on large-scale structured patient data, enabling them to convert longitudinal patient trajectories into generalizable representations for diverse clinical prediction tasks. Despite their effectiveness, FEMRs remain black-box models, raising concerns about bias, interpretability, and clinical trust. To address this, we propose the first token-level explainability approach for FEMRs. We train a Transformer-based surrogate model on input-output pairs from the FEMR across two prediction tasks, approximating its behavior while preserving temporal dynamics. We identify the most influential tokens, providing insights into how FEMRs leverage different aspects of patient history for predictions. To evaluate clinical relevance, we introduce a novel clinical alignment metric that quantifies the correspondence between the surrogate model's key tokens and clinically validated features. Our results demonstrate that the surrogate closely approximates FEMR predictions and that token-level explanations align well with clinical knowledge, offering a practical framework for interpretable and trustworthy clinical AI.
