Search papers, labs, and topics across Lattice.
This paper introduces Domain-Aware Task Prompt Representation Learning (DATPRL-IR), a novel method for multi-domain all-in-one image restoration (AiOIR). The approach uses task and domain prompt pools, combined via a prompt composition mechanism (PCM), to create domain-aware task representations. Experiments demonstrate that DATPRL-IR outperforms state-of-the-art image restoration methods and exhibits strong generalization across domains.
Achieve state-of-the-art image restoration across diverse image domains by adaptively composing task and domain prompts learned from multimodal LLMs.
Recently, significant breakthroughs have been made in all-in-one image restoration (AiOIR), which can handle multiple restoration tasks with a single model. However, existing methods typically focus on a specific image domain, such as natural scene, medical imaging, or remote sensing. In this work, we aim to extend AiOIR to multiple domains and propose the first multi-domain all-in-one image restoration method, DATPRL-IR, based on our proposed Domain-Aware Task Prompt Representation Learning. Specifically, we first construct a task prompt pool containing multiple task prompts, in which task-related knowledge is implicitly encoded. For each input image, the model adaptively selects the most relevant task prompts and composes them into an instance-level task representation via a prompt composition mechanism (PCM). Furthermore, to endow the model with domain awareness, we introduce another domain prompt pool and distill domain priors from multimodal large language models into the domain prompts. PCM is utilized to combine the adaptively selected domain prompts into a domain representation for each input image. Finally, the two representations are fused to form a domain-aware task prompt representation which can make full use of both specific and shared knowledge across tasks and domains to guide the subsequent restoration process. Extensive experiments demonstrate that our DATPRL-IR significantly outperforms existing SOTA image restoration methods, while exhibiting strong generalization capabilities. Code is available at https://github.com/GuangluDong0728/DATPRL-IR.
