S4oP: Operator-level Pruning of Structured State Space Models for Resource-Constrained Devices

2026-06-16

AI Summary

This work addresses the high computational and memory demands of Structured State Space Models (e.g., S4/S4D), which, despite their effectiveness in modeling long-range dependencies, hinder deployment on resource-constrained devices. The study presents the first systematic exploration of operator-level structured pruning tailored for S4/S4D, introducing an incremental mask generation strategy, a joint accuracy-latency monitoring mechanism, and a unified training-evaluation framework. By alternately applying structured pruning and fine-tuning, the method achieves efficient model compression. Experiments demonstrate that up to 70% of operators can be pruned across multiple benchmarks with negligible performance degradation, substantially reducing inference latency and offering a practical pathway toward lightweight deployment of state space models.

Abstract

Structured State Space Models (SSMs), including the S4 and S4D architectures, have recently emerged as powerful alternatives to attention-based models for capturing long-range dependencies in sequential data. Despite their strong empirical performance, deploying these models in time- and resource-constrained settings remains challenging due to their computational and memory demands. In this paper, we propose a novel incremental, operator-level pruning approach for S4- and S4D-based models that significantly reduces inference cost while preserving predictive performance. To the best of our knowledge, this is the first work to systematically investigate structured operator pruning for SSMs. Our method progressively prunes model operators by interleaving structured masking with fine-tuning, while jointly monitoring accuracy and inference latency. We implement this approach within a unified training and evaluation framework that enables systematic exploration of efficiency-accuracy trade-offs. Experiments across multiple benchmark datasets show that pruning up to 70% of the model operators preserves the performance of the original models in most cases, while substantially reducing inference latency. These results demonstrate that structured operator pruning is an effective and previously unexplored strategy for improving the efficiency of SSMs and facilitate their deployment in practical, resource-constrained scenarios.

Problem

• Structured State Space Models
• operator-level pruning
• resource-constrained devices
• inference latency
• model efficiency

Innovation

• structured pruning
• operator-level pruning
• State Space Models
• S4
• efficient inference