Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a lightweight RGB-guided hyperspectral super-resolution (HSR) framework that effectively addresses the challenges of cross-resolution misalignment in dual-camera systems. By employing cross-modal flow alignment and model-based Gram-Schmidt orthogonalization fusion, the method enhances reconstruction accuracy while maintaining a low computational footprint, allowing for flexibility across various spectral supports and scale factors. Experimental results demonstrate significant improvements in reconstruction accuracy compared to existing learned fusion methods, validating the approach under real-world conditions with genuine cross-sensor differences.
Achieving superior hyperspectral image reconstruction without the computational burden of traditional learned methods, this framework adapts seamlessly to varying sensor configurations.
Compact snapshot hyperspectral cameras provide rich instantaneous spectral measurements for ground-level machine vision, but at lower spatial resolution than standard RGB cameras. RGB-guided hyperspectral super-resolution (HSR) addresses this limitation by transferring spatial detail from a high-resolution RGB guide to a low-resolution hyperspectral image (HSI). These dual-camera systems are typically in a horizontal rig geometry, requiring cross-camera image alignment due to different fields of view. However, residual misregistration can inject spurious high-frequency details. Existing learned unaligned-fusion methods are usually trained for a fixed spectral support and spatial scale factors and can be computationally demanding, limiting their flexibility across sensors. We propose a lightweight and interpretable RGB-guided HSR framework combining cross-modal flow alignment with model-based Gram-Schmidt orthogonalization fusion. The method first warps the RGB guide onto the HSI grid, then estimates an energy-based confidence weight map by measuring local alignment reliability. This map is then used both in a weighted least-squares spectral regression and in a gated fusion between the super-resolved estimate and an HSI-preserving estimate. Unlike existing learned methods, the proposed framework has a low computational footprint and supports VIS-NIR spectral supports and scale factors without retraining. Experiments on the Real benchmark show that the proposed method improves reconstruction accuracy over learned fusion baselines while remaining substantially faster. On a 34-frame sequence acquired with our real RGB-HSI dual-camera setup, a reduced-resolution quantitative evaluation validates the method under genuine cross-sensor radiometric, noise, and geometric differences, while native-resolution qualitative results demonstrate deployment on the full 51-band VIS-NIR acquisition.