Search papers, labs, and topics across Lattice.
This paper introduces 4DGS-WAM, an object-centric world action model that utilizes a 4D Gaussian Splatting representation to effectively separate dynamic objects from static backgrounds in visual data. By leveraging past observations of static content, the model enhances future-state predictions of dynamic objects while minimizing redundant processing of the static environment. Experimental results on the KITTI-MOT dataset demonstrate significant improvements in short-horizon prediction and past reconstruction compared to traditional 2D WAMs.
By reusing static background information, 4DGS-WAM achieves more efficient and accurate predictions of dynamic object behavior in complex scenes.
Current world action models (WAMs) typically operate on 2D visual data. These models can achieve exceptional visual quality, but they lack explicit spatial structure for individual objects and repeatedly process redundant background content. Although point clouds can represent the world in 3D space, they can be difficult to align and accumulate across viewpoints. In this paper, we leverage an explicit 4D Gaussian Splatting (4DGS) representation that separately models dynamic objects and the static background of a scene. For dynamic objects, we use a policy model to predict future actor actions and a world model to predict transformations of their observed Gaussian splats. The static background need not be regenerated for future states, as much of it has already been observed in past frames. This forms an object-centric world action model, which we name 4DGS-WAM. It lifts 2D observations into a persistent 4D representation so that previously observed static content can be reused during future prediction. Future-state extrapolation can then focus on modeling the evolution of dynamic objects. Experiments on KITTI-MOT evaluate short-horizon prediction and past reconstruction.
