Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces POPSICLE, a comprehensive benchmark suite designed for segmentation and localization tasks in cryo-electron tomography (cryoET), addressing the critical need for standardized evaluation in this field. By leveraging the CryoET Data Portal, POPSICLE encompasses a diverse range of eukaryotic and prokaryotic samples, facilitating robust comparisons across various machine learning models. Baseline experiments reveal significant discrepancies in model performance across tasks, highlighting the necessity for tailored benchmarks that reflect the unique challenges of cryoET data analysis.
Substantial variations in model performance across cryoET tasks reveal the inadequacy of existing benchmarks and the urgent need for tailored evaluation frameworks.
Cryo-electron tomography (cryoET) has emerged as a powerful tool in structural and cellular biology by enabling direct visualization of macromolecular structures within intact cells, thereby linking molecular architecture to cellular organization in a native context. Realizing the full potential of cryoET, however, increasingly depends on advances in computational analysis, particularly machine learning (ML), to interpret its complex and information-rich data. Despite rapid progress, ML development for cryoET remains bottlenecked by the lack of standardized, well-annotated benchmarks. Existing evaluations are typically small, task-specific, and are assembled in isolation, limiting robust comparisons across methods. Here, we present POPSICLE, a benchmark suite for cryoET segmentation and macromolecular localization built from the CryoET Data Portal - an open, ML-ready repository of tomographic data, metadata, and annotations. POPSICLE spans eukaryotic and prokaryotic systems, both purified and fully in situ samples, and dense voxel-wise segmentation as well as sparse localization tasks. Built on a living data resource, it can expand as new datasets and annotations become available. Baseline experiments reveal substantial variation in model rankings across tasks, underscoring the need for benchmarks tailored to the unique characteristics of cryoET rather than evaluation practices adapted from adjacent biomedical imaging domains. POPSICLE thus provides an open and extensible foundation for reproducible ML evaluation in cryoET.