Search papers, labs, and topics across Lattice.
The paper introduces StableI2I, a no-reference image-to-image evaluation framework that explicitly measures content fidelity and pre-post consistency by assessing semantic correspondence and spatial structure preservation. StableI2I-Bench, a new benchmark, is introduced to evaluate MLLMs on these fidelity and consistency assessment tasks. Experiments show StableI2I provides accurate and interpretable evaluations, correlating well with human judgment, making it a practical tool for diagnosing content consistency in I2I systems.
Existing image-to-image evaluations miss a critical aspect: whether the output image actually preserves the content of the input.
In most real-world image-to-image (I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we propose StableI2I, a unified and dynamic evaluation framework that explicitly measures content fidelity and pre--post consistency across a wide range of I2I tasks without requiring reference images, including image editing and image restoration. In addition, we construct StableI2I-Bench, a benchmark designed to systematically evaluate the accuracy of MLLMs on such fidelity and consistency assessment tasks. Extensive experimental results demonstrate that StableI2I provides accurate, fine-grained, and interpretable evaluations of content fidelity and consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.
