Search papers, labs, and topics across Lattice.
This study evaluates the Segment Anything Model (SAM) by fine-tuning it on three challenging waste segmentation datasets, revealing that the SAM-ViT-H model significantly outperforms existing state-of-the-art methods, achieving a +30 increase in Intersection over Union (IoU) scores. Despite SAM's initial limitations in accuracy across diverse research fields, the findings underscore its potential as a foundational model for waste segmentation tasks. The results indicate that fine-tuning SAM is essential for enhancing its generalization capabilities in practical applications.
Fine-tuning SAM can boost waste segmentation performance by over 30 IoU points, challenging the notion that it should be overlooked.
Meta AI has recently released the Segment Anything Model (SAM), which demonstrates exceptional zero-shot image segmentation performance across various tasks with remarkable accuracy. Despite its inability to provide accurate segmentation across multiple research fields, SAM still serves as a valuable starting point for supporting the segmentation pipeline process, particularly for tasks that require extensive and senior skills annotations. This study aims to evaluate the generalization of SAM and fine-tuning SAM models using three waste segmentation datasets. Although they are captured from real scenes as SAM was pretrained on, these datasets present several challenges, including occlusions, deformable objects, transparency, and objects easily confused with backgrounds. In our findings, the fine-tuned SAM-ViT-H model outperforms the state-ofthe-art Zerowaste, and TACO datasets with a significant increase of +30 in IoU, and it closely approaches performance levels of TrashCan 1.0, with only a -1.44 difference. After evaluating these popular waste datasets, it became evident that fine-tuning SAM as a foundational model is a crucial step for providing better generalization for downstream waste segmentation tasks. Therefore, SAM should not be disregarded or wasted.
