Search papers, labs, and topics across Lattice.
This paper introduces PlankFormer, a novel framework for automating plankton instance segmentation by addressing the challenges of limited annotated datasets and the complexity of distinguishing plankton from debris. The authors generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto varied backgrounds and employ a Vision Transformer (ViT) with a Mask2Former decoder, enhanced by a Masked Autoencoder (MAE) for self-supervised pre-training. Experimental results show that PlankFormer significantly outperforms traditional methods like Mask R-CNN, particularly in environments with high debris density, achieving high-precision segmentation with reduced reliance on manual annotations.
Synthetic training images and a self-supervised architecture enable PlankFormer to achieve unprecedented accuracy in plankton segmentation amidst challenging debris-laden environments.
Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automating the segmentation of plankton from crowded images is crucial, however, it faces two major challenges: (i) the scarcity of pixel-level annotated datasets and (ii) the difficulty of distinguishing plankton from debris and overlapping individuals using conventional CNN-based methods. To address these issues, we propose PlankFormer, a novel framework for plankton instance segmentation. First, to overcome the data shortage, we introduce a method to generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto diverse backgrounds, including those created by generative models. Second, we propose a segmentation model utilizing a Vision Transformer (ViT) backbone with a Mask2Former decoder. To robustly capture the global structural features of plankton against occlusion and debris, we employ a Masked Autoencoder (MAE) for self-supervised pre-training on unlabeled individual images. Experimental results on real-world datasets demonstrate that our method significantly outperforms conventional methods, such as Mask R-CNN, particularly in challenging environments with high debris density. We demonstrate that our synthetic training strategy and MAE-based architecture enable high-precision segmentation with requiring less manual annotations for individual plankton images.
