Search papers, labs, and topics across Lattice.
This paper introduces STEDiff, a training-free method that enhances semantic representations in the text-embedding space for text-to-image generation. By leveraging the [EOT] token to strengthen sub-sentence semantics and incorporating a novel semantic enhancement loss to enforce spatial constraints, STEDiff significantly improves the alignment of complex prompts with generated images. Evaluations on T2I-CompBench show that STEDiff achieves superior semantic consistency and generation integrity compared to existing methods that rely on fine-tuning or layout priors.
STEDiff enhances text-to-image alignment without the need for costly fine-tuning, achieving remarkable semantic consistency even in complex prompts.
Although pretrained text-to-image (T2I) generation models can produce high-quality images, they often fail to faithfully reflect the semantic intent of complex prompts due to stochastic noise and inherent model limitations. This issue frequently manifests as the model overlooking specific objects or failing to correctly bind attributes to their corresponding entities, a challenge referred to as semantic alignment. Unlike existing approaches that rely on computationally expensive fine-tuning or labor-intensive layout priors, we propose STEDiff, a training-free method designed to enhance semantic representations directly within the text-embedding space. Specifically, we introduce a method that primarily leverages the [EOT] token to strengthen the relevant semantics of sub-sentences and then replaces the corresponding tokens in the original prompt. Furthermore, a novel semantic enhancement loss is incorporated to enforce spatial constraints, ensuring that the semantics of each entity are precisely mapped to their respective image regions. Extensive quantitative and qualitative evaluations on the T2I-CompBench demonstrate that our method notably improves semantic consistency and generation integrity in complex scenarios.
