Search papers, labs, and topics across Lattice.
This paper introduces a SAM-based annotation pipeline to generate dense, pixel-level semantic segmentation labels from bounding box annotations in the Zenseact Open Dataset (ZOD). They process 100,000 frames and curate a 2,300-frame subset to establish a reliable baseline for training segmentation models. Experiments with transformer-based CLFT and CNN-based DeepLabV3+ architectures demonstrate the effectiveness of the generated annotations, achieving up to 48.1% mIoU on ZOD and 77.5% mIoU on the Iseauto platform, while also addressing class imbalance issues.
Turning bounding box annotations into dense segmentation masks with SAM unlocks the potential of large, sparsely labeled datasets for training autonomous driving systems.
Dense semantic segmentation is essential for autonomous driving, yet many multi-modal datasets lack pixel-level annotations. The Zenseact Open Dataset (ZOD) provides rich multi-sensor data but only bounding-box labels, limiting its use for segmentation research. Our primary contribution is a Segment Anything Model (SAM)-based annotation pipeline that produces dense, pixel-level annotations for ZOD by converting bounding boxes into semantic masks. In this pilot study, we process over 100,000 frames and manually curate a 2,300-frame subset (36% acceptance rate) to establish a reliable baseline. Using these annotations, we evaluate transformer-based CLFT and CNN-based DeepLabV3+ architectures across diverse weather conditions, achieving up to 48.1% mIoU with CLFT-Hybrid. To address extreme class imbalance, where pedestrians, cyclists, and signs constitute less than 1% of pixels, we explore specialized models targeting rare classes. We further validate the pipeline on the Iseauto autonomous-vehicle platform, achieving 77.5% mIoU, and show that SAM-derived representations transfer effectively across sensor configurations via bidirectional transfer learning. All code and annotations are released to support reproducible research.
