Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Customized Concept Embedding Diffusion (CCE-Diffusion) framework to improve Foreground Conditioned Outpainting (FCO) by addressing the misalignment between text-derived concept embeddings and visual instances. The authors implement a CCE-Module that customizes embeddings to enhance the prominence of foreground objects while minimizing artifacts in synthesized backgrounds. Evaluations show that CCE-Diffusion significantly reduces these artifacts, making it a versatile enhancement for existing FCO methods.
Artifacts in synthesized backgrounds can be drastically reduced, enhancing the visual quality of outpainted images and foreground prominence.
To showcase products, merchants often incur substantial costs creating high-quality display images. Foreground Conditioned Outpainting (FCO) meets this demand, allowing users to create desired backgrounds for foreground instances at a low cost by adjusting the text prompt. However, existing text-driven FCO methods exhibit critical flaws in their outputs, most notably the presence of artifacts, which refer to regions in the synthesized background that share the same semantics as the foreground instance. Such artifacts diminish the object's prominence and degrade image quality. We attribute the issue to the misalignment between the given instance and text-derived concept embeddings. To address this, we propose the Customized Concept Embedding Diffusion (CCE-Diffusion) framework. Its core is a CCE-Module to customize concept embeddings, bridging the gap between generic noun semantics and a specific visual instance. An Instance-Aware Loss guides the module's optimization, while a Semantic-Preserving Prompt Template prevents customized embeddings from distorting other words in the prompt. Both qualitative and quantitative evaluations demonstrate that CCE-Diffusion significantly reduces artifacts in the outputs. As a plug-and-play component, the CCE-Module can integrate with various FCO methods, enhancing their performance.