Search papers, labs, and topics across Lattice.
This paper introduces GAUGE, a benchmark designed to evaluate the physical fidelity of numerical simulators and generative video world models against real-world physics. By employing 22 controlled task families that encompass various physical processes, the authors benchmark existing engines like Isaac Sim and Genesis, revealing significant discrepancies in their performance, particularly in scenarios involving impulsive contact and rapid motion. The findings highlight that while video world models can mimic expected trajectory forms, they often fail to accurately capture critical physical parameters, underscoring the need for improved simulation fidelity in embodied intelligence applications.
No physics engine is uniformly faithful, with critical failures in simulating impulsive contact and rapid textile motion revealed by the GAUGE benchmark.
Physics engines facilitate large-scale training and evaluation for embodied intelligence, while generative video world models are emerging as implicit simulators of future states and interactions. However, existing evaluations of physical fidelity are often conducted in isolation and rely heavily on perceptual similarity or human judgments, providing limited insight into which physical principles or parameters are violated. We introduce GAUGE, a real-world-grounded diagnostic benchmark for jointly evaluating how numerical simulators and generative video world models reproduce or deviate from real-world physics. It comprises 22 controlled task families covering rigid bodies, flexible cables, textiles, and volumetric deformable objects. Grounded in real-world trajectories and paired with calibrated physical metadata, uncertainty annotations, and task-specific observables, these tasks cover fundamental physical processes including collision, friction, momentum transfer, oscillation, self-contact, and deformation across diverse materials and conditions. We benchmark Isaac Sim, Genesis, and Newton on 14 task families using generalized trajectory errors, and evaluate 6 image-to-video models on 5 rigid-body tasks by testing physical-law consistency and the temporal stability of inferred parameters. Our results reveal no uniformly faithful physics engine, with the largest discrepancies arising in impulsive contact, rapid textile motion, and volumetric deformation. We further find that video world models can produce trajectories with the expected equation form while recovering incorrect accelerations, momentum transfer, and oscillation timing. GAUGE lays the groundwork for developing more physically faithful simulators and world models for embodied intelligence.
