Search papers, labs, and topics across Lattice.
This paper investigates self-supervised multi-image super-resolution (MISR) for camera array images, addressing limitations of existing supervised and self-supervised methods. It proposes a novel "Multi-to-Single-Guided Multi-to-Multi" SSL framework that combines the strengths of both Multi-to-Single and Multi-to-Multi approaches. The framework incorporates a dual Transformer network tailored for camera-array SR to improve high-frequency detail recovery, demonstrating superior performance on both synthetic and real-world datasets.
Camera array super-resolution gets a boost: a new self-supervised method leverages both multi-image-to-single-image and multi-image-to-multi-image techniques to generate sharper, more detailed images.
Conventional multi-image super-resolution (MISR) methods, such as burst and video SR, rely on sequential frames from a single camera. Consequently, they suffer from complex image degradation and severe occlusion, increasing the difficulty of accurate image restoration. In contrast, multi-aperture camera-array imaging captures spatially distributed views with sampling offsets forming a stable disk-like distribution, which enhances the non-redundancy of observed data. Existing MISR algorithms fail to fully exploit these unique properties. Supervised MISR methods tend to overfit the degradation patterns in training data, and current self-supervised learning (SSL) techniques struggle to recover fine-grained details. To address these issues, this paper thoroughly investigates the strengths, limitations and applicability boundaries of multi-image-to-single-image (Multi-to-Single) and multi-image-to-multi-image (Multi-to-Multi) SSL methods. We propose the Multi-to-Single-Guided Multi-to-Multi SSL framework that combines the advantages of Multi-to-Single and Multi-to-Multi to generate visually appealing and high-fidelity images rich in texture details. The Multi-to-Single-Guided Multi-to-Multi SSL framework provides a new paradigm for integrating deep neural network with classical physics-based variational methods. To enhance the ability of MISR network to recover high-frequency details from aliased artifacts, this paper proposes a novel camera-array SR network called dual Transformer suitable for SSL. Experiments on synthetic and real-world datasets demonstrate the superiority of the proposed method.
