Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ReSem-Face, a cascaded diffusion framework designed to enhance identity preservation in face inpainting, particularly under large occlusions and conflicting text prompts. By incorporating an identity-conditioned semantic prior and leveraging multiple reference images, the method effectively distills and reconstructs essential identity features, guiding the diffusion process for improved semantic coherence. Experiments reveal that ReSem-Face significantly outperforms existing baselines in maintaining identity fidelity and enhancing text-based editing quality on benchmark datasets like CelebAHQ-IDI-5 and VGGFace2.
Identity preservation in face inpainting can be dramatically improved with a novel multi-reference approach that stabilizes reconstruction under severe occlusions.
Face inpainting with diffusion models has recently achieved impressive visual quality, yet preserving identity fidelity under significant occlusion and conflicting text guidance remains a major challenge. To address this issue, we present Reference Semantic Inpainting for Face (ReSem-Face), a cascaded diffusion framework that introduces an explicit identity-conditioned semantic prior for multi-reference face inpainting. Our approach distills representative identity features from multiple references to reconstruct missing semantic regions, which then guide the diffusion process through a multi-stream conditioning architecture. This design provides strong semantic constraints when pixels are absent and stabilizes identity reconstruction while remaining compatible with prompt-driven edits. Experiments on CelebAHQ-IDI-5 and VGGFace2 demonstrate that ReSem-Face yields more reliable identity-preserving completion under severe semantic masks and improves text-controlled editing quality compared with representative baselines.