Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces VMetaphor-Bench, the first benchmark specifically designed to evaluate the capability of text-to-image (T2I) models in generating visual metaphors by blending elements from distinct domains. The benchmark consists of 1,500 curated visual metaphors organized into various categories and prompts, and employs a hybrid evaluation framework that assesses metaphorical fidelity through a combination of multiple-choice questions and dimension-based scoring. Findings reveal that even leading T2I models exhibit significant limitations in compositional structuring and cross-domain mapping, underscoring the need for further research in visual metaphor generation.
Even top-performing text-to-image models struggle with generating coherent visual metaphors, revealing a critical gap in their creative capabilities.
Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce visual metaphors, images that convey abstract ideas by combining elements from two distinct domains, remains largely unexamined. To bridge this gap, we introduce VMetaphor-Bench, the first benchmark for evaluating visual metaphor generation in T2I models. It comprises 1,500 visual metaphors curated from real-world creative imagery, organized into three levels and ten categories, with each sample paired with two prompts of differing specificity. For evaluation, we develop a hybrid framework within an MLLM-as-judge paradigm, combining a multiple-choice question (MCQ) based protocol of 9,594 questions across four levels of metaphorical fidelity with a dimension-based scoring protocol along three perceptual dimensions. Extensive evaluation of 11 representative T2I models reveals that even the strongest proprietary models struggle with compositional structuring and cross-domain mapping, key aspects of metaphorical expression, highlighting visual metaphor generation as an important frontier for future T2I research.