Search papers, labs, and topics across Lattice.
This paper evaluates the understanding of similes in text-to-image (t2i) models by introducing a scalable framework that combines a controlled simile dataset with automatic grounding metrics and text encoder layer analysis. The findings highlight that current t2i models frequently misinterpret metaphorical vehicles, leading to systematic failures in visual grounding. By identifying these failure patterns, the research opens avenues for improving the interpretation of figurative language in AI-generated imagery.
Text-to-image models consistently misinterpret similes, revealing a critical gap in their understanding of figurative language that could undermine their effectiveness in creative applications.
Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.
