Search papers, labs, and topics across Lattice.
This paper introduces URNet, a Unified Reparameterized Network that integrates RGB and depth inputs for semantic segmentation using a single encoder, addressing inefficiencies in traditional dual-encoder approaches. By employing a reparameterization strategy and a Linear Gated Attention module within its architecture, URNet enhances cross-modal feature extraction and fusion while minimizing computational redundancy. Experimental results across various benchmarks reveal that URNet achieves state-of-the-art performance in RGB-D segmentation with improved efficiency compared to existing methods.
URNet achieves state-of-the-art RGB-D semantic segmentation performance while significantly reducing computational overhead by unifying feature extraction and fusion in a single encoder.
Previous RGB-D semantic segmentation methods commonly employ dual encoders to separately process RGB and depth inputs, followed by dedicated modules for cross-modal feature fusion. However, such designs often inadequately capture depth representations and consequently limit effective cross-modal interaction, while the additional encoder branch introduces redundant computation that hinders lightweight execution. To tackle these challenges, we propose URNet, a Unified Reparameterized RGB-D Network that performs simultaneous multi-modal feature extraction and cross-modal fusion within a single encoder. Specifically, we adopt a reparameterization strategy to compact the network architecture and facilitate fast inference. Within each Reparameterized Block (RepBlock), a Linear Gated Attention (LGA) module is introduced to fully exploit complementary RGB and depth cues across different feature scales. Furthermore, considering that decoder design has been relatively underexplored in existing RGB-D segmentation models, we develop a concise yet effective universal decoder, termed the Pyramid Merging Decoder (PMD). Extensive experiments on multiple RGB-D segmentation benchmarks demonstrate that URNet achieves state-of-the-art performance while maintaining high efficiency. Code will be available at https://github.com/Wild-Stephen/URNet.
