Search papers, labs, and topics across Lattice.
This paper introduces FaceMoE, a novel Mixture of Experts (MoE) architecture tailored for low-resolution face recognition (LR-FR), addressing the challenges posed by poor feature extraction and the domain gap between high-resolution and low-resolution images. By employing multiple specialized feed-forward network experts and a dynamic top-k router, FaceMoE enhances resolution-aware feature extraction while mitigating catastrophic forgetting during fine-tuning. Experimental results across eleven diverse datasets reveal that FaceMoE outperforms existing state-of-the-art methods, demonstrating its effectiveness in tackling LR-FR challenges.
FaceMoE achieves unprecedented performance in low-resolution face recognition by leveraging a dynamic Mixture of Experts architecture that promotes specialization and preserves pretrained knowledge.
Low-resolution face recognition (LR-FR) remains a challenging task due to poor feature extraction and aggregation, as probe images often contain limited identity information resulting from extreme degradations such as blur, occlusion, and low contrast. Additionally, the domain gap between high-resolution (HR) gallery images and low-resolution (LR) probe images poses a significant challenge. A single feature encoder struggles to generalize effectively across both domains when fine-tuned on an LR dataset, and this issue is further magnified by catastrophic forgetting. To address these challenges, we propose FaceMoE, an effective adaptation of Mixture of Experts (MoE) transfomer architecture for low-resolution face-recognition . Specifically, we introduce multiple specialized feed-forward network (FFN) experts and incorporate a top-k router, which dynamically assigns tokens to appropriate experts. This design emergently promotes specialization across experts for different semantic regions of the face, which enables FaceMoE to perform resolution-aware feature extraction. Moreover, the top-k router facilitates sparse expert activation, enabling the model to preserve pretrained knowledge when finetuned on a LR dataset, while increasing model capacity without proportional computational overhead. FaceMoE is trained with a combined face recognition loss, router z-loss, and load balancing loss to ensure expert specialization and stable training. To the best of our knowledge, this is the first work leveraging MoE for LR-FR. Extensive experiments across eleven datasets, spanning HR, mixed-quality, and LR benchmarks, demonstrate that FaceMoE significantly outperforms state-of-the-art methods. Code: https://github.com/Kartik-3004/FaceMoE
