Search papers, labs, and topics across Lattice.
This study systematically compares 2D and 3D masked autoencoders (MAE-2D vs. MAE-3D) for self-supervised learning in fluorescence microscopy, revealing that MAE-3D consistently outperforms its 2D counterparts on various downstream single-cell tasks. By integrating cross-modal supervision with a pretrained protein language model (ESM2), the researchers achieve significant performance gains, particularly in protein-protein interaction and localization tasks. Key innovations such as channel cross-attention and frequency-domain regularization are highlighted as essential for effectively utilizing 3D spatial context, leading to state-of-the-art results in the evaluated metrics.
MAE-3D not only surpasses 2D methods in single-cell tasks but also sets new benchmarks for protein localization and interaction, showcasing the power of 3D modeling in microscopy.
Self-supervised learning in fluorescence microscopy often relies on 2D projections, despite the inherently three-dimensional nature of cells. We present a systematic comparison of 2D and 3D masked autoencoders (MAE-2D vs. MAE-3D) on volumetric microscopy data. Under matched architectures and training protocols, MAE-3D consistently outperforms 2D max-projection and slice-based variants on downstream single-cell tasks. We further align visual representations with a pretrained protein language model (ESM2) and show that cross-modal supervision yields larger gains for volumetric models. Channel cross-attention and frequency-domain regularization are critical for leveraging 3D spatial context. On a protein--protein interaction task, MAE-3D achieves a ROC--AUC of 0.865, outperforming prior methods by up to +0.025. For protein localization, our best 3D model attains state-of-the-art AUC$_{\text{micro}}$ (0.952) and F1$_{\text{micro}}$ (0.742), improving over previous approaches by +0.003 and +0.010 absolute, respectively. Overall, these results demonstrate the advantages of native 3D modeling and multimodal alignment for representation learning in single-cell microscopy.
