Search papers, labs, and topics across Lattice.
The authors introduce Qwen-Image-Bench, a new text-to-image benchmark designed with professional artists to evaluate real-world fidelity and creative generation capabilities beyond simple text-image alignment. They organize the benchmark into a hierarchical taxonomy with 5 pillars, 23 sub-capabilities, and 56 verifiable rubrics, using 1000 stratified prompts. A unified judge model, Q-Judger, is trained on Qwen3.6-27B with supervision from 80 professional annotators to score images across all 56 facets, providing fine-grained diagnostics.
Existing text-to-image benchmarks miss the mark on real-world artistic creation, but Qwen-Image-Bench finally provides a creator-centric evaluation that reliably distinguishes state-of-the-art models.
Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple text-image alignment can no longer satisfy users'pressing demands for faithful real-world reconstruction and genuine creative expression. Existing benchmarks, however, remain anchored in these foundational criteria and do not yet capture the nuanced capabilities that matter in authentic artistic practice, making it difficult to reliably distinguish state-of-the-art T2I models. To address the gap, we introduce Qwen-Image-Bench, a creator-centric benchmark co-designed with professional artists and grounded in real-world creation scenarios. Qwen-Image-Bench enriches conventional evaluation with two application-driven dimensions: Real-world Fidelity and Creative Generation. Drawing on the staged reasoning inherent in professional artistic workflows, we organize these five pillars into a top-down hierarchical taxonomy that further decomposes into 23 second-level sub-capabilities and 56 third-level verifiable rubrics. To ensure broad coverage, we curate 1000 stratified prompts with each prompt jointly exercising more than four fine-grained facets across multiple pillars. We train a unified judge model Q-Judger based on Qwen3.6-27B, supervised by 80 professional annotators from global art academies under blind labeling and triple-review protocols, that scores every image across all 56 verifiable facets, producing fine-grained, rubric-grounded, and fully attributable diagnostics rather than a single opaque score. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the two application-driven dimensions of Real-world Fidelity and Creative Generation where existing benchmarks provide little insight, while also providing a trustworthy optimization signal for production-level T2I development.
