Search papers, labs, and topics across Lattice.
This paper introduces two novel difference-driven gating methods鈥擣eature-difference gating (FDG) and Entropy-difference gating (EDG)鈥攆or adaptive feature fusion in U-Net decoders, focusing on the difference between high-level and low-level features. By leveraging the absolute difference and information entropy, these methods create coupled gating maps that enhance the fusion process, leading to improved performance across various tasks such as medical image segmentation and speech separation. The experiments demonstrate that both FDG and EDG surpass traditional attention-based fusion techniques, with EDG yielding the best results, suggesting a significant advancement in multi-scale feature integration strategies.
Rethinking feature fusion, this study reveals that focusing on the differences between feature streams can dramatically enhance U-Net performance across diverse applications.
The U-Net style models have been widely used in many applications. A critical step in these models is to reconstruct the lower-level features using a top-down decoder. This reconstruction requires precise fusion of high-level semantics and low-level details. Existing attention-based fusion methods typically derive attention weights from the top-down decoder features (global) alone or the correlation between the top-down decoder features and the bottom-up encoder features (local), then modulate the encoder features using these weights. In this work, we explore a different paradigm: deriving attention weights from the difference between the two feature streams. To this end, we propose two difference-based gating approaches: Feature-difference gating (FDG), which directly uses the absolute difference between global and local features to generate adaptive gating maps, and Entropy-difference gating (EDG), which measures the representational certainty of each stream via information entropy and uses their signed entropy difference to derive the attention weights. Both methods produce coupled gating maps that simultaneously modulate the global and local features. Experiments on different tasks including medical image segmentation, remote sensing image cloud removal and speech separation showed that both methods outperformed existing attention-based fusion methods, and EDG performed better. The results suggested a new paradigm for multi-scale feature fusion in the U-Net style structures.
