Search papers, labs, and topics across Lattice.
This paper introduces MiM-DiT, a novel all-in-one image restoration framework leveraging a dual-level Mixture-of-Experts (MoE) architecture integrated with a pre-trained diffusion model. The Inter-MoE layer handles major degradation types by adaptively combining expert groups, while the Intra-MoE layer selects specialized sub-experts for fine-grained variations. Experiments demonstrate state-of-the-art performance across multiple image restoration tasks, showcasing the framework's ability to handle complex, real-world corruptions through coarse-grained adaptation and fine-grained modulation.
A dual-level Mixture-of-Experts architecture can effectively unify diverse image restoration tasks within a single diffusion transformer model, achieving state-of-the-art results.
All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task.
