Search papers, labs, and topics across Lattice.
This paper introduces CIGTSurv, a tri-modal framework for survival prediction that effectively integrates clinical information with pathology images and genomic data. By employing a dual-level interaction mechanism鈥攍ocal prototype association for token-level correspondences and global feature alignment for cross-modal consistency鈥攖he model addresses the challenges posed by the discrete and sparse nature of clinical data. Extensive experiments across five TCGA cancer cohorts show that CIGTSurv achieves state-of-the-art performance in survival prediction, highlighting the importance of clinical information in multimodal learning.
Clinical information can significantly enhance survival prediction accuracy when effectively integrated with multimodal data, as shown by CIGTSurv's state-of-the-art results.
Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, despite its critical role in reflecting a patient's overall health, remains underutilized due to its discrete, sparse, and low-dimensional nature. Furthermore, the inherent heterogeneity across these modalities pose significant challenges in modeling cross-modal interactions. In this paper, we propose CIGTSurv, a Clinical Information Guided Tri-modal framework for Survival prediction. Specifically, we first design a holistic text template and use pretrained foundation models to transform clinical tabular data into high-dimensional tokenized embeddings. Using clinical information as an anchor, we then introduce a dual-level interaction mechanism: 1) a local prototype association (LPA) module based on cross-attention to explicitly learn token-level correspondences between different modalities, and 2) a global feature alignment (GFA) loss based on Maximum Mean Discrepancy (MMD) to implicitly enhance cross-modal distribution consistency. Extensive experiments on five TCGA cancer cohorts demonstrate that CIGTSurv achieves state-of-the-art (SOTA) survival prediction performance. Our source code is publicly available at https://github.com/Daijing-ai/CIGT-Surv.git.
