Reload-Mamba: Hierarchical Anti-Dilution State-Space Modeling for Multi-Class Semantic Segmentation

2026-06-16

AI Summary

This work addresses the degradation of boundary and fine-detail responses in Mamba-based models for multi-class semantic segmentation, which arises from sequential state propagation and leads to reduced segmentation accuracy. To mitigate this issue, the authors propose Reload-Mamba, a novel framework that introduces an anti-dilution mechanism into dense prediction tasks for the first time. It restores detailed representations through a top-down hierarchy across three decoder stages, leveraging boundary-supervised local detail priors, class-uncertainty-aware reloading gates, and a multi-level hierarchical reloading structure. Integrated with a ConvNeXt-Tiny encoder, four-directional Mamba scanning, and pixel-wise directional attention, the model achieves state-of-the-art performance on ADE20K (48.9% mIoU), Cityscapes (83.2%), and PASCAL VOC 2012 (87.8%, a +2.2% improvement), demonstrating its effectiveness in recovering boundary fidelity and fine-grained details.

Abstract

Mamba-based state space models offer linear-time long-range modeling for high-resolution dense prediction, but sequential state-space propagation can attenuate boundary-sensitive and detail-sensitive responses that are critical in multi-class semantic segmentation. We propose Reload-Mamba, a semantic segmentation framework that addresses this propagation-induced response dilution through three segmentation-specific designs: (i) a boundary-supervised local detail prior that is explicitly trained with ground-truth boundary masks to identify regions requiring response restoration; (ii) a class-uncertainty-aware Reload Gate that incorporates per-pixel class entropy from a pre-reload auxiliary head as an additional gating signal, a formulation that is informative only under multi-class dense prediction; and (iii) a hierarchical multi-level Reload mechanism that applies anti-dilution refinement at three decoder levels and fuses the restored representations top-down. Built upon a ConvNeXt-Tiny encoder with a multi-scale decoder and four-directional Mamba scanning with pixel-wise directional attention, Reload-Mamba achieves 47.9% single-scale (48.9% multi-scale) mIoU on ADE20K and 83.2% single-scale mIoU on Cityscapes. With ResNet-101 + COCO pre-training under the standard DeepLab-style protocol, Reload-Mamba reaches 87.8% mIoU on PASCAL VOC 2012 val. Controlled ablations show that each of the three segmentation-specific designs contributes beyond a direct port of the prior anti-dilution architecture proposed for binarization, cumulatively improving over the direct-port baseline by +2.2 mIoU on ADE20K.

Problem

• semantic segmentation
• state-space models
• response dilution
• boundary sensitivity
• multi-class

Innovation

• Reload-Mamba
• state-space model
• semantic segmentation
• anti-dilution
• hierarchical refinement