Search papers, labs, and topics across Lattice.
This paper introduces M$^3$-Gen, a framework that generates gene expression profiles by conditioning a Generative Adversarial Network (GAN) on histopathology images and clinical metadata. By employing contrastive learning to create a unified latent representation, M$^3$-Gen effectively produces biologically coherent gene expression data while maintaining interpretability through an attention-based mechanism. Evaluations on the TCGA dataset reveal that the generated profiles are not only realistic but also meaningful, allowing researchers to trace the influence of specific image regions on gene expression outcomes.
M$^3$-Gen reveals how multimodal integration can generate interpretable gene expression profiles, bridging the gap between clinical imaging and molecular data.
Integrating heterogeneous biomedical data, including clinical metadata, histopathology images, and molecular profiles, is crucial for comprehensive disease understanding. However, gene expression data acquisition remains constrained by high costs and privacy concerns, limiting its use in multimodal research and AI-driven applications. We present MultiModal Molecular Generation (M$^3$-Gen), a novel framework for the generation of gene expression profiles by conditioning a Generative Adversarial Network on histopathology images and clinical metadata. M$^3$-Gen learns a unified latent representation from the clinical variables and the images, leveraging contrastive learning, and exploits the embeddings of the two modalities to guide a generative model in producing biologically coherent gene expression profiles. Evaluations on the TCGA dataset demonstrate that M$^3$-Gen generates realistic and functionally meaningful gene expression data. Importantly, by integrating multiple modalities in an attention-based mechanism, M$^3$-Gen provides intrinsic explainability: it allows the identification of which regions of the histopathology images most strongly influenced the generation of specific gene expression profiles, making the model's decisions interpretable by design.
