Search papers, labs, and topics across Lattice.
This paper introduces a novel framework, SPaRa-DCAL, for subject-driven personalized text-to-image generation that integrates stage-aware low-rank adaptation and distribution-calibrated candidate selection. By addressing the limitations of existing methods, which often fail to account for varying capacity requirements across different denoising stages, the authors demonstrate that their approach significantly enhances the quality of generated images while maintaining identity consistency and representation diversity. Experimental results reveal that DCAL improves key metrics such as 1-LPIPS and CLIP-I, underscoring the importance of evaluating personalized generation through a multifaceted lens beyond mere identity metrics.
Personalized text-to-image generation can achieve unprecedented quality by combining stage-aware adaptation with intelligent candidate selection, revealing a nuanced trade-off between identity consistency and representation diversity.
Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images while preserving subject identity, following novel text prompts, and maintaining sample diversity. Existing optimization-based methods instantiate subject adaptation through full fine-tuning, textual embedding optimization, or low-rank parameter updates; PaRa further constrains personalization from the perspective of parameter rank reduction. However, a uniform low-rank constraint or a uniform adapter strength cannot explicitly distinguish the capacity requirements of different denoising stages. Moreover, inference-time candidate selection driven mainly by identity similarity may compress the selected samples in the visual representation space. We decompose the problem into two complementary components: SPaRa denotes training-side stage-aware low-rank adaptation, DCAL denotes inference-side distribution-calibrated candidate selection, and SPaRa-DCAL denotes the combined framework. Theoretical analysis shows that timestep-dependent scaling controls the effective perturbation magnitude of a low-rank adapter, while identity-biased candidate selection restricts the radius of selected features around the reference center under explicit conditions. Auditable experiments under the SDXL and DreamBooth 30-subject protocol show that DCAL improves 1-LPIPS, CLIP-I, DINO-I, and CLIP-T on a fixed LoRA candidate pool, while revealing a clear trade-off with CLIP/DINO pairwise diversity and pairwise LPIPS. These results indicate that personalized generation should be evaluated through identity consistency, text alignment, and representation diversity rather than identity metrics alone.
