Search papers, labs, and topics across Lattice.
PicoSAM3, a 1.3M parameter promptable segmentation model, was developed and optimized for real-time, in-sensor execution on devices like the Sony IMX500. It combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. The model achieves 65.45% mIoU on COCO and 64.01% on LVIS, and the INT8 quantized version maintains accuracy while achieving 11.82 ms latency on the IMX500.
You can now run high-quality, promptable segmentation models directly on a vision sensor in real-time, thanks to a 1.3M parameter model that outperforms larger SAM-based baselines.
Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3 M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82 ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.
