Search papers, labs, and topics across Lattice.
This paper introduces TerraVis, a novel framework designed to evaluate world-grounded visual consistency in text-to-image generation, addressing limitations of existing metrics that fail to capture real-world plausibility failures. TerraVis employs a multi-stage MLLM-based workflow to detect and quantify object-, interaction-, and scene-level violations across 18 categories, achieving superior correlation with human judgments compared to prior methods. Benchmark results reveal that even models excelling in conventional metrics exhibit significant world-consistency issues, underscoring the importance of this complementary evaluation dimension.
State-of-the-art text-to-image models, despite photorealism, still exhibit pervasive world-consistency failures that existing metrics miss.
Recent text-to-image models have made substantial progress in photorealism, aesthetics, and text-image alignment. Yet visually appealing images can still violate real-world plausibility, exhibiting malformed object structures, impossible anatomy, physically implausible interactions, or inconsistent spatial relationships. Such failures are not well captured by existing fidelity, aesthetics, preference, or alignment metrics. To address this gap, we introduce TerraVis, a framework for evaluating world-grounded visual consistency in generated images. TerraVis defines a structured taxonomy of world-consistency violations spanning object-, interaction-, and scene-level failures, and employs a multi-stage evaluation framework to identify and quantify them. Given an image, TerraVis first uses an MLLM to assess its eligibility for evaluation, then detects violations across 18 taxonomy-defined types and classifies them as minor or major to derive an overall world-consistency score. Across diverse open-source and proprietary text-to-image models on two widely used benchmarks, TerraVis achieves the strongest correlation with human judgments of world consistency among existing metrics. Our benchmark results further show that models that achieve strong performance on conventional metrics can still exhibit substantial world-consistency failures. These findings highlight world consistency as a complementary evaluation dimension and demonstrate that TerraVis enables systematic quantification, diagnosis, and comparison of such failures. Our code is publicly available at https://github.com/ShyFoo/TerraVis.
