Search papers, labs, and topics across Lattice.
The paper investigates whether computationally intensive deep learning-based foundation models are necessary for achieving state-of-the-art performance on downstream single-cell RNA sequencing (scRNA-seq) tasks. They demonstrate that simple, interpretable pipelines using normalization and linear methods achieve comparable or superior performance to foundation models like TranscriptFormer on cell-type classification, disease-state prediction, and cross-species learning, particularly on out-of-distribution tasks. This suggests that linear representations can effectively capture cell identity, challenging the necessity of complex deep learning models.
Single-cell foundation models get schooled by simple linear methods, especially when generalizing to unseen cell types and organisms.
Single-cell RNA sequencing (scRNA-seq) data exhibit strong and reproducible statistical structure. This has motivated the development of large-scale foundation models, such as TranscriptFormer, that use transformer-based architectures to learn a generative model for gene expression by embedding genes into a latent vector space. These embeddings have been used to obtain state-of-the-art (SOTA) performance on downstream tasks such as cell-type classification, disease-state prediction, and cross-species learning. Here, we ask whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations. Using simple, interpretable pipelines that rely on careful normalization and linear methods, we obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data. Our findings highlight the need for rigorous benchmarking and suggest that the biology of cell identity can be captured by simple linear representations of single cell gene expression data.
