Search papers, labs, and topics across Lattice.
This study introduces a novel explainable framework that integrates relevance propagation and concept discovery to elucidate how morphological features in H&E images contribute to spatial transcriptomics (ST) predictions. By employing a ViT-based architecture, the framework not only predicts clinically relevant ST signatures but also generates a relevance-based concept atlas that connects molecular phenotypes to tissue morphology. The results show substantial spatial heterogeneity in colorectal cancer subtypes, with high weighted F1 scores indicating the framework's effectiveness in stratifying patient outcomes based on predicted gene expression profiles.
A relevance-based concept atlas reveals how tissue morphology directly influences spatial transcriptomics predictions, enhancing interpretability in pathology.
Recent advances in pathology foundation models have enabled accurate prediction of spatial transcriptomics (ST) from routine H&E images. However, existing explainability methods for vision transformer (ViT)-based models are largely limited to local heatmaps and do not reveal how morphological concepts contribute to ST predictions. Here, we introduce an explainable framework that combines relevance propagation and concept discovery to link transcriptional programs to tissue morphology. We developed a ViT-based framework for virtual ST from H&E images that combines ViT-aware layer-wise relevance propagation with relaxed archetypal TopK sparse autoencoder-based concept discovery. This approach provides both local explanations and global insights into the morphological patterns associated with transcriptional programs. We applied the framework to colorectal cancer ST data from the HEST-1k cohort and evaluated its generalizability in TCGA COAD. Our architecture accurately predicts clinically relevant ST signatures and accompanying molecular phenotypes. Measured and predicted gene expression profiles reveal substantial spatial heterogeneity of the colorectal cancer subtypes iCMS2 and iCMS3 across a large number of samples. Spatially resolved and aggregated iCMS classification achieve weighted F1 scores of 0.872 and 0.819 (0.770 in TCGA COAD), respectively, and both stratify patient outcome. Beyond prediction, our framework establishes a relevance-based concept atlas linking molecular phenotypes to histopathological representations. Comparison of activation- with relevance-derived concepts demonstrates that relevances provide a more direct link between tissue morphology and downstream predictions. We establish a general strategy for concept-based explanation of spatial prediction, and our framework is readily applicable to a broad range of ViT-based pathology models.
