Search papers, labs, and topics across Lattice.
This paper introduces Position Embedding Modulation with Occlusion-Level Attention (PEMOLA), an innovative module designed to enhance panoptic segmentation by effectively modeling occlusions. By training an occlusion classifier on the COCO-OLAC dataset, the authors derive occlusion-level attention that guides spatial processing and integrates occlusion priors into position embeddings. Extensive evaluations on both COCO-OLAC and the newly annotated Cityscapes-OLAC dataset reveal that PEMOLA significantly improves segmentation quality while maintaining low computational costs, underscoring the critical role of occlusion modeling in complex scene understanding.
Occlusion-aware attention can dramatically enhance panoptic segmentation performance without incurring significant computational costs.
Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose \textbf{P}osition \textbf{E}mbedding \textbf{M}odulation with \textbf{O}cclusion-\textbf{L}evel \textbf{A}ttention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into transformer-based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC dataset. The classifier derives the occlusion-level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo-wei/PEMOLA.
