Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the assumption that final-layer embeddings are optimal in single-cell foundation models (scFMs) by evaluating layer-wise representations from scFoundation and Tahoe-X1 on trajectory inference and perturbation response prediction. They find that optimal layers are task-dependent, with trajectory inference peaking at 60% depth and perturbation optima shifting significantly across cell states. Surprisingly, first-layer embeddings outperform deeper layers in quiescent cells, indicating that hierarchical feature abstraction is not always beneficial.
Single-cell foundation models hide their most biologically relevant information in intermediate layers, blowing up the common assumption that final-layer embeddings are always best.
Current single-cell foundation model benchmarks universally extract final layer embeddings, assuming these represent optimal feature spaces. We systematically evaluate layer-wise representations from scFoundation (100M parameters) and Tahoe-X1 (1.3B parameters) across trajectory inference and perturbation response prediction. Our analysis reveals that optimal layers are task-dependent (trajectory peaks at 60% depth, 31% above final layers) and context-dependent (perturbation optima shift 0-96% across T cell activation states). Notably, first-layer embeddings outperform all deeper layers in quiescent cells, challenging assumptions about hierarchical feature abstraction. These findings demonstrate that"where"to extract features matters as much as"what"the model learns, necessitating systematic layer evaluation tailored to biological task and cellular context rather than defaulting to final-layer embeddings.