Search papers, labs, and topics across Lattice.
PhysFlow tackles physical inconsistencies in generative video by decoupling generation into physics-aware optical flow modeling followed by flow-conditioned appearance synthesis. To support explicit physical dynamics, the authors design a physics-aware attention module conditioned on material and motion attributes, backed by PhysVideo鈥攁 new benchmark of 50K physics-engine-simulated sequences rendered with 3D Gaussian Splatting. The approach significantly improves the physical plausibility and motion controllability of generated dynamics over monolithic baselines while preserving high visual fidelity.
Video models consistently fail at realistic dynamics because they conflate appearance with physical laws, a failure resolved here by synthesizing material-conditioned optical flow fields before rendering pixels.
Video generation models have recently attracted substantial attention for their ability to generate visually compelling videos, yet ensuring physically consistent and plausible dynamics still remains a fundamental challenge, driving a growing line of research on physical realism in video generation. To address this challenge, motivated by the fact that physical regularities are primarily encoded in motion patterns, we propose PhysFlow, a novel two-stage framework for improving the physical plausibility of generated videos by decomposing video generation into motion-aware optical flow generation followed by motion-conditioned appearance synthesis. Specifically, PhysFlow consists of a physics-aware optical-flow video generator called PA-Flow and a flow-guided video generator called FlowRender. During the first stage, PA-Flow employs a physics-aware attention module to model how motion attributes and material properties influence global motion and local deformation, respectively, and generates an optical flow video as an explicit representation of motion. In the second stage, FlowRender leverages the decoupled motion representation as guidance to synthesize realistic textures and appearances, ultimately producing the final physically plausible video. To further support model training with explicit physical supervision, we construct PhysVideo, a physics-based video dataset generated with a physics engine and 3D-GS rendering, containing 10K foreground objects and 50K realistic video sequences with annotations of motion and material properties. Extensive experiments demonstrate that our proposed PhysFlow generates videos with superior physical plausibility while maintaining high visual fidelity compared with existing methods.
