Search papers, labs, and topics across Lattice.
This paper benchmarks six foundation models (CellPoseSAM, CellSAM, μSAM, SAM, SAM2, SAM3) on diverse microscopy datasets for cell, nucleus, and organoid segmentation. It introduces Automatic Prompt Generation (APG), a novel instance segmentation strategy to improve SAM-based microscopy foundation models. APG enhances μSAM's performance, achieving results comparable to CellPoseSAM, and offers insights for adapting SAM-style models to microscopy.
A new prompting strategy closes the gap between general-purpose and specialized cell segmentation models, suggesting a path to more efficient adaptation.
Cell segmentation is a fundamental task in microscopy image analysis. Several foundation models for cell segmentation have been introduced, virtually all of them are extensions of Segment Anything Model (SAM), improving it for microscopy data. Recently, SAM2 and SAM3 have been published, further improving and extending the capabilities of general-purpose segmentation foundation models. Here, we comprehensively evaluate foundation models for cell segmentation (CellPoseSAM, CellSAM, $μ$SAM) and for general-purpose segmentation (SAM, SAM2, SAM3) on a diverse set of (light) microscopy datasets, for tasks including cell, nucleus and organoid segmentation. Furthermore, we introduce a new instance segmentation strategy called automatic prompt generation (APG) that can be used to further improve SAM-based microscopy foundation models. APG consistently improves segmentation results for $μ$SAM, which is used as the base model, and is competitive with the state-of-the-art model CellPoseSAM. Moreover, our work provides important lessons for adaptation strategies of SAM-style models to microscopy and provides a strategy for creating even more powerful microscopy foundation models. Our code is publicly available at https://github.com/computational-cell-analytics/micro-sam.
