Search papers, labs, and topics across Lattice.
This paper introduces RainDancer, a novel RGB-Event video deraining framework that utilizes a decompose-before-interact approach to effectively separate rain and background components in dynamic scenes. By employing a rain-oriented spiking neural network to capture the unique dynamics of rain motion, the method achieves enhanced structure preservation and rain suppression through component-level fusion of semantically aligned representations. Experimental results on both synthetic and real datasets show that RainDancer significantly outperforms existing methods in terms of quantitative metrics, visual quality, and robustness in downstream perception tasks.
RainDancer achieves superior deraining performance by intelligently separating rain and background components before fusion, addressing the challenges of dynamic rainy scenes.
Video deraining aims to recover clean visual content from rainy videos for reliable perception under adverse weather. Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB-only restoration remains ambiguous in dynamic rainy scenes, where rain streaks, textures, boundaries, motion, and occlusions may share similar visual patterns. Event cameras provide complementary motion-sensitive cues with high temporal resolution, but event streams also contain sensor noise and background-triggered responses, so direct RGB-Event fusion may introduce cross-modal interference. To address this issue, we propose RainDancer, a progressive RGB-Event video deraining framework based on a decompose-before-interact paradigm. The core idea is to separate rain and background components within each modality before cross-modal interaction. In the RGB branch, frame features are progressively decomposed into rain and background representations. In the event branch, a rain-oriented spiking neural network module captures sparse and bursty event dynamics associated with rain motion. Component-level fusion is then performed between semantically aligned representations for structure preservation and rain suppression. We further introduce event-domain supervision to regularize sparse event reconstruction, structural consistency, and gradient orientation. Experiments on synthetic and real RGB-Event video deraining datasets demonstrate superior quantitative performance, visual quality, and downstream perception robustness. Code is available at https://github.com/AE86-plus/RainDancer.
