Search papers, labs, and topics across Lattice.
This paper introduces MOSH-WM, a mask-grounded soft-Hamiltonian world model that enhances object-centric video forecasting by explicitly linking state representations to image support from entity slots. By employing a frozen video-slot encoder and utilizing spatial moments of mask-owned support to form a canonical state, the model achieves significant reductions in prediction error on benchmark datasets OBJ3D and CLEVRER. The results demonstrate a 25.0% reduction in LPIPS and a 33.7% reduction in spatial MSE on OBJ3D, indicating improved accuracy in long-term video predictions compared to existing methods.
Achieving a 25% reduction in prediction error, MOSH-WM redefines the landscape of object-centric video forecasting by grounding state representations in visual support.
Object-centric world models forecast future videos by evolving a set of entity slots, but the variables receiving dynamics supervision are often unconstrained visual features. We introduce \method{}, a mask-grounded soft-Hamiltonian world model that makes its position-like state explicitly depend on slot-owned image support. A frozen video-slot encoder produces slots and masks; spatial moments of mask-owned support form a canonical state $Q$, temporal differences form $P$, and a learned energy supplies a soft directional bias to a bounded learned increment. Decoder-relevant appearance and identity are stored separately in a causal visual context. A gated composer and bounded residual then combine this context with the propagated phase state to reconstruct decoder-compatible slots. On OBJ3D, given six observed frames and evaluated over the following 30 frames, \method{} reduces LPIPS by 25.0\% and spatial MSE by 33.7\% relative to the strongest object-centric baseline. On CLEVRER, given six observed frames and evaluated over the following ten frames, the corresponding reductions are 14.5\% and 18.7\%. Horizon-resolved visual and object-state measurements show that the complete model accumulates error more slowly throughout the 30-frame closed-loop rollout. Project page:https://github.com/moshwm-anon/-moshwm-anon.github.io.
Tsinghua AI