Search papers, labs, and topics across Lattice.
MambaRaw introduces a novel JPEG-conditioned metadata-based framework for efficient raw image reconstruction, specifically targeting high-resolution 4K images. By leveraging State Space Models (SSMs) and a Spatial-Energy Coupled Context Modeling mechanism, it significantly reduces computational costs associated with traditional attention mechanisms while enhancing feature representation. Experimental results demonstrate that MambaRaw achieves state-of-the-art performance, improving PSNR by 1.2–1.4 dB and decreasing coding latency by approximately 9% compared to existing baselines.
MambaRaw achieves a remarkable 1.4 dB increase in PSNR at low metadata bitrates while slashing coding latency by nearly 9%, setting a new benchmark in raw image reconstruction.
In-camera JPEG previews are ubiquitous in raw image formats and provide an sRGB reference at negligible storage cost. Although existing metadata-based reconstruction frameworks can exploit this side information when recovering raw images, their context models often become computationally expensive especially at high resolution, eg, 4K raw image, given that attention mechanisms scale quadratically with feature maps, hindering its practical application. To address these limitations, we propose MambaRaw, a JPEG-conditioned metadata-based raw image reconstruction framework that uses State Space Models (SSMs) to estimate entropy parameters efficiently. Our key contribution comprises a Spatial-Energy Coupled Context Modeling mechanism with two lightweight modules: (1) TileMambaBlock, which performs Mamba-style selective scanning only on information-dense tiles to improve the efficiency; and (2) Energy-Aware Refinement (EAR), an identity-initialized residual module that enhance feature representation to match the long-tail energy distribution of raw signals. Extensive experiments on three camera datasets (Sony, Olympus, Samsung) show consistent improvements over strong metadata-based baselines and set a new state of the art for JPEG-guided raw reconstruction with great efficiency. Notably, at low metadata bitrates, MambaRaw increases PSNR by 1.2--1.4 dB and reduces end-to-end coding latency by about 9%. Code is released at https://github.com/Peizeli1/MambaRaw.
Microsoft Research
Tsinghua AI