Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
HiPhy is a hierarchical reinforcement learning framework designed to imbue video generation models with physical plausibility, addressing the current limitation of generating videos that adhere to multiple concurrent physical laws. It achieves this by employing a dual-level objective that locally enforces temporal dynamics of individual principles and globally ensures scene coherence, outperforming prior methods on complex multi-principle scenarios. The framework is supported by a new 50K-prompt dataset and benchmark, MultiPhyBench, facilitating the development of more realistic and versatile world simulators.
Generating videos that simultaneously obey multiple interacting physical laws, like buoyancy and fluid dynamics, is now achievable with a novel RL framework that explicitly aligns temporal dynamics and global scene coherence.
Video generation models have achieved remarkable visual fidelity and have strong potential to become general-purpose world simulators. Despite this progress, they still fail to generate videos which adhere to laws of physics. The problem becomes even more apparent in realistic settings where multiple physical principles must work together within the same video; for example, "a balloon floating upward while steam rises from a pot" requires buoyancy and fluid dynamics to unfold coherently and simultaneously. Yet existing methods largely ignore multi-principle interactions, focusing on a single principle per video. We propose HiPhy (Hierarchical Physical Alignment), a reinforcement learning framework that grounds video generation in physical laws through a dual-level objective: locally enforcing the temporal dynamics of individual physical principles, and globally ensuring the physical and semantic coherence of the entire scene. To support multi-principle generation, we construct a 50K-prompt dataset and introduce a prompt benchmark MultiPhyBench, spanning a diverse range of co-occurring physical events. Our experiments show that HiPhy significantly outperforms prior methods and baselines, improving physical commonsense and semantic alignment significantly across various benchmarks, with the largest gains on scenes involving multiple concurrent physical principles where competing methods degrade most sharply.