Search papers, labs, and topics across Lattice.
This paper addresses the problem of hallucinated content in camera images due to the increasing integration of generative AI into camera image signal processors (ISPs). They propose a method to recover the "unhallucinated" version of the camera image by optimizing an image-specific MLP decoder with a modality-specific encoder, which can be applied post-capture without access to the camera ISP. The method achieves this with a small footprint (180KB) suitable for embedding as metadata within standard image formats.
Your camera's AI could be subtly rewriting reality, but this method lets you reverse the changes and see the "unhallucinated" original.
The ability of generative AI (GenAI) methods to photorealistically alter camera images has raised awareness about the authenticity of images shared online. Interestingly, images captured directly by our cameras are considered authentic and faithful. However, with the increasing integration of deep-learning modules into cameras'capture-time hardware -- namely, the image signal processor (ISP) -- there is now a potential for hallucinated content in images directly output by our cameras. Hallucinated capture-time image content is typically benign, such as enhanced edges or texture, but in certain operations, such as AI-based digital zoom or low-light image enhancement, hallucinations can potentially alter the semantics and interpretation of the image content. As a result, users may not realize that the content in their camera images is not authentic. This paper addresses this issue by enabling users to recover the'unhallucinated'version of the camera image to avoid misinterpretation of the image content. Our approach works by optimizing an image-specific multi-layer perceptron (MLP) decoder together with a modality-specific encoder so that, given the camera image, we can recover the image before hallucinated content was added. The encoder and MLP are self-contained and can be applied post-capture to the image without requiring access to the camera ISP. Moreover, the encoder and MLP decoder require only 180 KB of storage and can be readily saved as metadata within standard image formats such as JPEG and HEIC.
