Search papers, labs, and topics across Lattice.
This paper introduces RS4D, a novel instance segmentation method for remote sensing that leverages distilled state space modeling (SSM) to achieve linear computational complexity, addressing the inefficiencies of traditional ViT-based models. By employing an adaptive noise and masking knowledge distillation training method, the authors effectively compress knowledge from self-attention mechanisms into a more efficient linear state space. Experimental results demonstrate that RS4D achieves an 8x reduction in parameters and a 9x reduction in FLOPs while maintaining or surpassing the accuracy of existing ViT- and CNN-based methods across multiple benchmark datasets.
Achieving linear-time efficiency in remote sensing instance segmentation could revolutionize how we deploy vision models in resource-constrained environments.
The computational complexity of Transformers scales quadratically with the number of tokens, which significantly constrains the efficiency of vision models, particularly recent ViT-based foundation models in dense prediction tasks. Instance segmentation, a typical dense visual prediction task in the remote sensing field, faces similar challenges. In this paper, inspired by the recent advances of knowledge distillation in large language models, we introduce RS4D - a new remote sensing instance segmentation method with linear computational complexity, which addresses the inefficiency of long sequence modeling through distilled state space modeling (SSM). We propose an adaptive noise and masking knowledge distillation training method for pre-training lightweight SSM backbones, which effectively compresses knowledge from the vast self-attention space into a compact, dense linear state space. We also design a remote sensing image instance segmentation architecture based on this lightweight visual encoder, where we explore variants of three different backbones and two segmentation heads. Extensive experiments are conducted on multiple benchmark datasets, including SSDD, WHU, and NWPU. Compared to ViT-based approaches, our proposed SSM backbone achieves an 8x reduction in parameters and a 9x reduction in FLOPs while maintaining comparable or superior accuracy to both ViT- and CNN-based instance segmentation methods. The implementation codes have been publicly available at https://github.com/QinzheYang/RS4D.
