Search papers, labs, and topics across Lattice.
This paper introduces a dictionary-guided, coefficient-domain framework for infrared-visible (IR-VIS) image fusion when the infrared modality is missing. The method learns a shared convolutional dictionary for both modalities, transfers visible image coefficients to pseudo-IR coefficients guided by a frozen large language model, and adaptively fuses the representations at the atom level. Experiments demonstrate improved perceptual quality and downstream detection performance compared to pixel-space generative approaches in missing-IR scenarios.
By operating in the coefficient domain with a shared dictionary and LLM guidance, this method achieves interpretable and controllable IR-VIS fusion even when infrared data is missing.
Infrared-visible (IR-VIS) image fusion is vital for perception and security, yet most methods rely on the availability of both modalities during training and inference. When the infrared modality is absent, pixel-space generative substitutes become hard to control and inherently lack interpretability. We address missing-IR fusion by proposing a dictionary-guided, coefficient-domain framework built upon a shared convolutional dictionary. The pipeline comprises three key components: (1) Joint Shared-dictionary Representation Learning (JSRL) learns a unified and interpretable atom space shared by both IR and VIS modalities; (2) VIS-Guided IR Inference (VGII) transfers VIS coefficients to pseudo-IR coefficients in the coefficient domain and performs a one-step closed-loop refinement guided by a frozen large language model as a weak semantic prior; and (3) Adaptive Fusion via Representation Inference (AFRI) merges VIS structures and inferred IR cues at the atom level through window attention and convolutional mixing, followed by reconstruction with the shared dictionary. This encode-transfer-fuse-reconstruct pipeline avoids uncontrolled pixel-space generation while ensuring prior preservation within interpretable dictionary-coefficient representation. Experiments under missing-IR settings demonstrate consistent improvements in perceptual quality and downstream detection performance. To our knowledge, this represents the first framework that jointly learns a shared dictionary and performs coefficient-domain inference-fusion to tackle missing-IR fusion. The source code is publicly available at https://github.com/harukiv/DCMIF.
