Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models

2026-06-18

AI Summary

Existing vision Mamba models suffer significant performance degradation during token compression due to the neglect of spatial structure. This work proposes STORM, a plug-and-play framework that, for the first time, incorporates spatial awareness into the compression process. By leveraging structured spatial partitioning and local topological constraints, STORM preserves grid neighborhood consistency without requiring any training. The module seamlessly integrates into existing compression pipelines and substantially enhances reconstruction fidelity. Experimental results demonstrate that STORM achieves state-of-the-art, training-free performance across multiple vision Mamba backbones: it boosts VMamba’s Top-1 accuracy by up to 63.3% and limits PlainMamba’s drop to merely 1.0%, matching the performance level of Vision Transformers.

Abstract

Mamba demonstrates strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit a severe performance collapse. We attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism. In this work, we propose STORM, a spatial-aware token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to maintain both grid topology and neighborhood coherence. As a plug-and-play module, STORM equips existing reduction pipelines with explicit spatial awareness without any training. Empirical results demonstrate that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. Notably, STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3\% in top-1 accuracy. Meanwhile, STORM incurs only a 1.0\% accuracy drop on PlainMamba, achieving performance comparable to ViT.

Problem

• token reduction
• spatial awareness
• visual state space models
• structural integrity
• Mamba

Innovation

• spatial-aware reduction
• Vision Mamba
• token pruning
• structured compression
• training-free