Search papers, labs, and topics across Lattice.
This paper introduces SWAAP, a novel two-stage data poisoning framework that targets learned world models in model-based learning agents. By leveraging first-order bilevel optimization, SWAAP identifies and manipulates fine-tuning trajectories to induce low-return behaviors while maintaining proximity to clean dynamics. The evaluation demonstrates that SWAAP effectively degrades performance across various continuous-control tasks while evading existing detection defenses, highlighting a critical vulnerability in world-model adaptation processes.
SWAAP can stealthily degrade the performance of world models by manipulating only a small fraction of training data, revealing a significant vulnerability in model-based learning systems.
Model-based learning agents use learned world models to predict future states, plan actions, and adapt to new environments. However, the process of updating world models from collected experience creates a training-time attack surface: adversarially poisoned fine-tuning trajectories can manipulate the learned dynamics and thereby corrupt downstream planning. In this paper, we propose SWAAP, the first two-stage data poisoning framework for learned world models. In the first stage, SWAAP identifies a harmful target world model that induces low-return behavior under planning while remaining close to clean dynamics, using first-order bilevel optimization enabled by a transition-gradient theorem. In the second stage, SWAAP realizes this target through stealth-constrained gradient matching, modifying only a limited fraction of fine-tuning transition targets so that the induced training gradients steer the victim model toward the adversarial target, while a prediction-error regularizer encourages the poisoned targets to remain close to the world model's natural approximation error. To assess attack stealthiness, we evaluate defenses and detectability across three stages of the poisoning pipeline: pre-training detection of poisoned transitions, robust training during fine-tuning, and test-time monitoring of the resulting world model. Across diverse continuous-control tasks, SWAAP causes substantial performance degradation while keeping poisoned transitions close to clean data and evading the evaluated non-adaptive residual/CUSUM/TRIM-style defenses. These results reveal a practical vulnerability in world-model adaptation pipelines and highlight the need for robustness methods that protect both world-model training data and learned dynamics.
