Search papers, labs, and topics across Lattice.
DiffWind introduces a physics-informed differentiable framework to model wind-driven object dynamics from video, representing wind as a grid-based physical field and objects as particle systems interacting via the Material Point Method (MPM). The framework jointly optimizes the spatio-temporal wind force field and object motion using differentiable rendering and simulation, constrained by the Lattice Boltzmann Method (LBM) to ensure physical validity. Experiments on a new dataset, WD-Objects, demonstrate superior reconstruction accuracy and simulation fidelity compared to existing dynamic scene modeling techniques.
Reconstructing and simulating wind-driven dynamics from video is now possible with a new differentiable framework that enforces fluid dynamics laws.
Modeling wind-driven object dynamics from video observations is highly challenging due to the invisibility and spatio-temporal variability of wind, as well as the complex deformations of objects. We present DiffWind, a physics-informed differentiable framework that unifies wind-object interaction modeling, video-based reconstruction, and forward simulation. Specifically, we represent wind as a grid-based physical field and objects as particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). To recover wind-driven object dynamics, we introduce a reconstruction framework that jointly optimizes the spatio-temporal wind force field and object motion through differentiable rendering and simulation. To ensure physical validity, we incorporate the Lattice Boltzmann Method (LBM) as a physics-informed constraint, enforcing compliance with fluid dynamics laws. Beyond reconstruction, our method naturally supports forward simulation under novel wind conditions and enables new applications such as wind retargeting. We further introduce WD-Objects, a dataset of synthetic and real-world wind-driven scenes. Extensive experiments demonstrate that our method significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening a new avenue for video-based wind-object interaction modeling.
