Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Cross-Stage Attention Propagation (CSAP), a novel decoder framework for semantic segmentation that reduces computational redundancy by computing attention only at the deepest feature scale and propagating it to shallower stages. By bypassing query-key computation at shallower stages, CSAP maintains multi-scale contextual reasoning with significantly reduced computational cost. CSAP-Tiny achieves state-of-the-art performance on ADE20K, Cityscapes, and COCO-Stuff 164K datasets with fewer FLOPs compared to existing lightweight semantic segmentation models like SegNeXt-Tiny.
Attention is often redundant across feature scales in semantic segmentation, but this work shows you can propagate attention maps from the deepest layer to shallower layers and maintain accuracy while slashing compute.
Recent lightweight semantic segmentation methods have made significant progress by combining compact backbones with efficient decoder heads. However, most multi-scale decoders compute attention independently at each feature scale, introducing substantial redundancy since the resulting attention distributions across scales are strongly correlated. We propose Cross-Stage Attention Propagation (CSAP), a decoder framework that computes attention at the deepest feature scale and propagates the resulting attention maps to shallower stages, bypassing query-key computation at those stages entirely. This design preserves multi-scale contextual reasoning while substantially reducing the decoder's computational cost. CSAP-Tiny achieves 42.9% mIoU on ADE20K with only 5.5 GFLOPs, 80.5% on Cityscapes with 21.5 GFLOPs, and 40.9% on COCO-Stuff 164K with 5.5 GFLOPs, surpassing SegNeXt-Tiny by +1.8% on ADE20K while requiring 16.8% fewer floating-point operations.