Search papers, labs, and topics across Lattice.
This study introduces BioM-JEPA, a joint-embedding predictive architecture that enhances the representation learning of single-cell transcriptomes by predicting aggregate representations of graph-connected gene blocks rather than individual genes. The method employs a student-teacher framework where the student infers target-block representations from remaining genes, while the teacher provides the corresponding targets, leading to embeddings with higher effective rank and improved performance across various diagnostics. Notably, BioM-JEPA achieved a significant increase in throughput compared to existing models, supporting the utility of graph-connected gene blocks in single-cell biology representation learning.
Achieving 5.75-fold higher fine-tuning throughput, BioM-JEPA redefines how we approach representation learning in single-cell transcriptomics.
Single-cell transcriptomes are sparse observations of coordinated biological programmes, yet most self-supervised models learn by reconstructing individual genes. Here we present BioM-JEPA, a joint-embedding predictive architecture that instead predicts aggregate representations of graph-connected gene blocks defined by protein-association and corpus-derived coexpression evidence. A student network infers each target-block representation from the remaining genes in a cell, while a slowly updated teacher supplies the corresponding target from the full observed gene set. Under the reported extraction procedure, block-level prediction produced embeddings with higher effective rank and weaker association with detected-gene depth in the tested diagnostics than token-prediction, random-block and reconstruction controls. Across CellBench tasks, frozen BioM-JEPA embeddings retained expression, pathway and neighbourhood information and achieved the lowest aggregate perturbation-response error among the evaluated models. Representation diagnostics were also consistent with canonical pancreatic programmes and compositional relationships between genetic perturbations. Linear attention avoids constructing a quadratic gene-by-gene attention matrix; in a matched one-epoch hPancreas experiment at batch size 8, BioM-JEPA provided 5.75-fold higher fine-tuning throughput and 3.76-fold higher held-out embedding throughput than scFoundation. Together, these results support graph-connected gene blocks as useful prediction units for JEPA-style representation learning in single-cell biology.
