Cross-Stage Attention Propagation for Efficient Semantic Segmentation
This work addresses the significant computational redundancy in existing lightweight semantic segmentation methods, which independently compute attention at each level of multi-scale decoders. To overcome this inefficiency, the authors propose a Cross-Stage Attention Propagation (CSAP) mechanism that computes attention only once at the deepest feature layer and efficiently propagates it to shallower layers, thereby eliminating redundant query-key operations while preserving multi-scale contextual modeling capability. Built upon CSAP, the lightweight model CSAP-Tiny achieves 42.9% mIoU on ADE20K with only 5.5 GFLOPs, outperforming SegNeXt-Tiny by 1.8% mIoU while reducing computational cost by 16.8%. This approach is the first to enable cross-stage sharing of attention distributions, effectively balancing efficiency and performance.