Search papers, labs, and topics across Lattice.
This paper introduces OmniPhys, a comprehensive benchmark of 1,551 samples designed to evaluate physical commonsense in text-to-image generation, addressing the limitations of existing benchmarks that fail to capture specific physical principles. By leveraging a Physical Knowledge Graph and aligning it with PhET simulations, the authors implement a dual-path verification protocol to rigorously assess the physical consistency of generated images. The proposed OmniPrompt framework optimizes prompt generation through iterative feedback aggregation, resulting in significant improvements in physical alignment across various text-to-image models, highlighting universal physical bottlenecks in current systems.
Text-to-image models struggle with physical commonsense, but OmniPhys reveals and addresses these critical flaws through a novel benchmarking and optimization approach.
While text-to-image models exhibit remarkable visual fidelity, they frequently violate fundamental physical commonsense. Existing benchmarks often rely on coarse-grained descriptions, failing to diagnose the mastery of specific physical principles. Moreover, the high stochasticity of generative processes causes current prompt optimization methods to suffer from gradient hallucinations, where optimizers are misled by transient visual artifacts rather than systemic flaws. To address these challenges, we introduce OmniPhys, a rigorous benchmark of 1,551 samples grounded in a Physical Knowledge Graph. By aligning PhET simulations with standard curricula, OmniPhys operationalizes a knowledge-to-scenario pipeline that performs diagnostic stress tests via a dual-path verification protocol. We further propose OmniPrompt, an iterative framework that treats physical alignment as a discrete optimization problem. For each query, OmniPrompt aggregates K stochastic images into a per-query feedback buffer. Across training, it further merges feedback from batches of B queries before each meta-policy update, filtering seed and query-local noise. Evaluations across 12 representative text-to-image models reveal universal physical bottlenecks. Results demonstrate that OmniPrompt significantly enhances physical consistency across diverse backbones, proving the transferability and efficacy of our evolved meta-policies. The code and data are available at https://github.com/zjukg/OmniPhys
