Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates diffusion sequence models for in-context meta-learning of robot dynamics, comparing them to Transformer-based deterministic models. They introduce inpainting diffusion (Diffuser) and conditional diffusion models (CNN and Transformer) to predict forward dynamics. Results show that diffusion models, particularly inpainting diffusion, significantly improve robustness under distribution shift, and can operate in real-time with warm-started sampling.
Diffusion models can dramatically improve the robustness of robot dynamics models to distribution shift, outperforming deterministic Transformers in challenging conditions.
Accurate modeling of robot dynamics is essential for model-based control, yet remains challenging under distributional shifts and real-time constraints. In this work, we formulate system identification as an in-context meta-learning problem and compare deterministic and generative sequence models for forward dynamics prediction. We take a Transformer-based meta-model, as a strong deterministic baseline, and introduce to this setting two complementary diffusion-based approaches: (i) inpainting diffusion (Diffuser), which learns the joint input-observation distribution, and (ii) conditioned diffusion models (CNN and Transformer), which generate future observations conditioned on control inputs. Through large-scale randomized simulations, we analyze performance across in-distribution and out-of-distribution regimes, as well as computational trade-offs relevant for control. We show that diffusion models significantly improve robustness under distribution shift, with inpainting diffusion achieving the best performance in our experiments. Finally, we demonstrate that warm-started sampling enables diffusion models to operate within real-time constraints, making them viable for control applications. These results highlight generative meta-models as a promising direction for robust system identification in robotics.