Search papers, labs, and topics across Lattice.
This paper introduces an innovative operator-level pruning method for Structured State Space Models (SSMs), specifically targeting the S4 and S4D architectures, to enhance their deployment in resource-constrained environments. By interleaving structured masking with fine-tuning, the approach systematically reduces inference costs while maintaining predictive accuracy, achieving up to 70% pruning of model operators. Experimental results across various benchmark datasets confirm that this method significantly decreases inference latency without compromising model performance, marking a significant advancement in the efficiency of SSMs.
Pruning up to 70% of operators in SSMs can maintain performance while slashing inference latency, unlocking their potential for real-world applications.
Structured State Space Models (SSMs), including the S4 and S4D architectures, have recently emerged as powerful alternatives to attention-based models for capturing long-range dependencies in sequential data. Despite their strong empirical performance, deploying these models in time- and resource-constrained settings remains challenging due to their computational and memory demands. In this paper, we propose a novel incremental, operator-level pruning approach for S4- and S4D-based models that significantly reduces inference cost while preserving predictive performance. To the best of our knowledge, this is the first work to systematically investigate structured operator pruning for SSMs. Our method progressively prunes model operators by interleaving structured masking with fine-tuning, while jointly monitoring accuracy and inference latency. We implement this approach within a unified training and evaluation framework that enables systematic exploration of efficiency-accuracy trade-offs. Experiments across multiple benchmark datasets show that pruning up to 70% of the model operators preserves the performance of the original models in most cases, while substantially reducing inference latency. These results demonstrate that structured operator pruning is an effective and previously unexplored strategy for improving the efficiency of SSMs and facilitate their deployment in practical, resource-constrained scenarios.
