Search papers, labs, and topics across Lattice.
This paper introduces a semi-supervised flow matching framework to fuse low-resolution mosaiced hyperspectral images with high-resolution panchromatic images for video-rate HR-HSI imaging. The method uses a two-stage training pipeline: first, an unsupervised prior network generates a pseudo HR-HSI, then a conditional flow matching model refines this estimate using a random voting mechanism. Experiments on benchmark datasets show significant performance gains over existing methods in both quantitative and qualitative metrics.
Achieve state-of-the-art hyperspectral image fusion by ditching handcrafted assumptions and protocols for a generalizable flow-matching framework.
Fusing a low resolution (LR) mosaiced hyperspectral image (HSI) with a high resolution (HR) panchromatic (PAN) image offers a promising avenue for video-rate HR-HSI imaging via single-shot acquisition, yet its severely ill-posed nature remains a significant challenge. In this work, we propose a novel semi-supervised flow matching framework for mosaiced and PAN image fusion. Unlike previous diffusion-based approaches constrained by specific protocols or handcrafted assumptions, our method seamlessly integrates an unsupervised scheme with flow matching, resulting in a generalizable and efficient generative framework. Specifically, our method follows a two-stage training pipeline. First, we pretrain an unsupervised prior network to produce an initial pseudo HR-HSI. Building on this, we then train a conditional flow matching model to generate the target HR-HSI, introducing a random voting mechanism that iteratively refines the initial HR-HSI estimate, enabling robust and effective fusion. During inference, we employ a conflict-free gradient guidance strategy that ensures spectrally and spatially consistent HR-HSI reconstruction. Experiments on multiple benchmark datasets demonstrate that our method achieves superior quantitative and qualitative performance by a significant margin compared to representative baselines. Beyond mosaiced and PAN fusion, our approach provides a flexible generative framework that can be readily extended to other image fusion tasks and integrated with unsupervised or blind image restoration algorithms.
