Search papers, labs, and topics across Lattice.
This paper reveals that world models, while beneficial for generating robot training data, can serve as a covert entry point for data poisoning attacks that compromise robotic learning pipelines. By injecting malicious prompts into ostensibly safe datasets, the authors demonstrate how these attacks can lead to the creation of dangerous synthetic training trajectories, ultimately resulting in unsafe robotic policies. The study highlights the effectiveness of these attacks against both action-conditioned and text-conditioned world models, emphasizing the urgent need for enhanced security measures in the integration of world models within robot learning systems.
World models can stealthily introduce data poisoning vulnerabilities that lead to unsafe robotic behaviors, even when trained on safe datasets.
World models have recently seen a rapid growth in both their popularity and capability as more data efficient tools for generating robot training data or simulating real world environments, with many works proposing their integration into the robot learning pipeline. While highly practical, in this work we demonstrate that world models introduce a uniquely stealthy and effective data poisoning entry point into the robot learning supply chain that can result in the deployment of unsafe or otherwise compromised robotic policies despite training on seemingly safe ground truth training data. In contrast to traditional data poisoning techniques which directly implant dangerous trajectories into sold or uploaded datasets, our novel attack methods inject malicious prompts or compromising transition dynamics into visibly safe teleoperated datasets which are only activated once fed through a world model as input. This can result in the generation of synthetic, dangerous robot training trajectories and subsequently unsafe or compromised robot policies. We demonstrate the effectiveness of our attacks against both state of the art action conditioned and text conditioned world models, showing a full end-to-end backdoor on a downstream DRL policy and a proof-of-concept for the VLA setting. Overall these findings necessitate research into more secure world models and reevaluating their position within the robot learning supply chain.
