Search papers, labs, and topics across Lattice.
This paper introduces CIS-Fuse, a novel spiking neural network architecture designed for infrared and visible image fusion (IVIF) that leverages a current injection spiking (CIS) operator to integrate information from both modalities at the membrane-potential level. By allowing both modalities to jointly influence the output before spike firing, CIS-Fuse effectively retains fine-grained responses while maintaining the energy efficiency characteristic of spiking neural networks. Experimental results demonstrate that CIS-Fuse achieves fusion quality comparable to state-of-the-art ANN methods while consuming approximately an order of magnitude less energy during inference.
Achieving state-of-the-art image fusion quality with spiking neural networks while slashing inference energy consumption by an order of magnitude.
Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content. While existing methods are largely built on artificial neural networks (ANNs), which densely compute over all activations, spiking neural networks (SNNs) communicate through sparse binary spikes and compute only where and when a spike occurs, offering a route to more energy-efficient fusion. However, directly applying SNNs to IVIF creates a fundamental tension: cross-modal fusion relies on fine-grained responses from both modalities, whereas binary spikes can discard complementary cues that remain below the firing threshold. The membrane potential retains these subthreshold responses before firing, letting both modalities jointly shape the output when integrated at this stage. Building on this, we propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level. At its core is the current injection spiking (CIS) operator, which injects one modality as a gated auxiliary current into the driving neuron of the other, so the two integrate before spike firing, with a per-channel learnable injection strength that adaptively regulates the modulation magnitude. Building on CIS, we construct a bidirectional cross-modal fusion (BCMF) module and deploy it on a dual-branch architecture with asymmetric stacking depths, where the two branches develop a clear functional specialization. Extensive experiments on four IVIF benchmarks and on downstream detection and segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo. Code will be released upon publication.
