Search papers, labs, and topics across Lattice.
This paper introduces a sparse and residual world model that selectively predicts changes in object states rather than re-predicting entire scenes, significantly enhancing accuracy and efficiency in physical prediction and control tasks. Evaluated on a MuJoCo tabletop pushing benchmark, the proposed model achieves 2.5 to 4.6 times greater accuracy with 8.6 to 11.1 times fewer parameters compared to traditional dense models, while also demonstrating robust transferability across varying object counts without retraining. The model not only reduces error accumulation during autoregressive rollouts but also enables successful planning in a sampling-based framework, marking a substantial advancement in object-centric manipulation.
By focusing on what changes rather than the entire scene, this model achieves unprecedented accuracy and efficiency in object manipulation tasks.
Monolithic world models predict the entire next state at every step, spending capacity re-predicting the static majority of a scene and injecting error into it. We ask whether explicitly modeling change (a per-object change gate plus a residual delta head that perturbs only the objects the gate flags) is a more effective and interpretable bias for physical prediction and control. On a MuJoCo tabletop pushing benchmark scaling from 3 to 8 objects, the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron at 8.6 to 11.1 times fewer parameters, sustains change-detection F1 of 0.80 to 0.87 where the dense baseline is degenerate, transfers across object counts with zero retraining (99.4 percent F1 retention), and reaches about 90 percent of its full-data accuracy with a quarter of the data. In autoregressive rollout it compounds far less error, hugging the no-motion floor while the dense model drifts. Finally, inside a sampling-based planner, prediction-only models fail (though a true-simulator oracle solves the task with the identical planner, confirming the planner is sound), but once featurized and trained for the states a planner visits, the sparse model begins to plan (0.23 plus or minus 0.06 success over three seeds) while the dense monolith stays at zero at every seed. Modeling what changes, rather than re-predicting the whole world, is a simple, effective bias for object-centric physical AI; code, data generators, and all checkpoints will be released upon publication.
