Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a two-stage framework, GenBloom, to align single white blood cell images with corresponding karyotype and somatic mutation data for improved hematological diagnosis. The framework first pretrains a vision transformer aggregator using iBOT on a large cohort, then aligns it with genetic data via supervised contrastive loss on AML patients. Results demonstrate that the genetically aligned patient encoder outperforms slide-level histopathology foundation models in hematological diagnostic tasks and enables effective disease and genetic alteration retrieval.
Aligning single-cell images with genetic data unlocks superior hematological diagnosis compared to traditional histopathology foundation models.
Multimodal alignment of histopathology encoders with transcriptomic and genomic data has been shown to significantly improve performance in downstream diagnostic tasks. Hematological cytology is unique in that visual single-cell evaluation is often paired with cytogenetics and molecular genetics for blood cancer diagnosis. In this study, we present a framework to align single white blood cell images with chromosomal aberrations (karyotype) and somatic mutations from targeted gene panels. Our training strategy follows a two-stage approach: (i) self-supervised, vision-only pretraining of a transformer aggregator using an iBOT head on a cohort of over 1500 patients, and (ii) genetic alignment via supervised contrastive loss on acute myeloid leukemia patients. Our genetically aligned patient encoder improves hematological diagnostic tasks, outperforming slide-level histopathology foundation models. Additionally, the model provides off-the-shelf retrieval capabilities for diseases and genetic alterations. Incorporating genetic data into patient encoders increases the quality of patient representations, providing a framework that aligns with clinical diagnostic workflows and paves the way for future multimodal hematology-specific AI. The code and model weights are available at https://github.com/marrlab/GenBloom.