Search papers, labs, and topics across Lattice.
This paper introduces Decoupled Guidance (DeGu), a novel framework for text-to-image personalization that disentangles the pathways for subject identity and scene context to mitigate the fidelity-editability trade-off caused by conditioning entanglement. By employing a spatial mixing mechanism, DeGu allows for independent control of subject fidelity and contextual editability, leading to enhanced personalization performance across various models. The results demonstrate that this approach not only improves the quality of generated images but also provides users with greater flexibility in balancing fidelity and editability during inference.
Conditioning entanglement in text-to-image models can be effectively resolved by decoupling subject and context pathways, leading to superior personalization outcomes.
Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text. However, most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway, forcing the two to compete for attention-map resources. We refer to this phenomenon as conditioning entanglement and show that it induces a fidelity-editability trade-off. We further provide causal evidence by replacing the target subject token with a generic subject token, which produces shifts in attention allocation and corresponding changes in context adherence. To this end, we propose Decoupled Guidance (DeGu), a plug-and-play framework that routes subject identity and scene context through two independent guidance streams. We further introduce a spatial mixing mechanism that dynamically fuses these streams, ensuring each operates within its semantically relevant region without interference. Furthermore, DeGu can be readily applied to existing personalization methods without modifying the underlying backbone models, consistently improving the overall personalization performance while enabling inference-time control over the fidelity-editability balance, across diverse methods and backbones, including flow-matching Diffusion Transformers (DiTs).
