Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to multimodal image fusion by utilizing a compact 1D token interface derived from a frozen pretrained image tokenizer, which allows for improved modeling of global appearance factors while preserving local detail through a 2D spatial pathway. The method, termed Selective Token Editing (STE), selectively updates a small set of critical tokens to enhance global coherence without altering the fusion backbone, thus avoiding additional losses. Experimental results across four benchmarks demonstrate that this approach significantly outperforms existing methods in both global coherence and local fidelity metrics.
A compact 1D token interface can dramatically enhance multimodal image fusion, achieving superior global coherence without sacrificing local detail.
Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/
