Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper investigates the action-semantic mismatch in world-model controllers caused by asynchronous real-world execution, moving beyond treating it as a mere disturbance. It identifies distinct failure modes in planning-based (TD-MPC2) and recurrent (DreamerV3) controllers: a future-action timeline mismatch for the former and misattributed transitions for the latter. Lightweight, architecture-specific interfaces are introduced to correct these mismatches without retraining, demonstrating robust performance across various asynchronous execution conditions.
Asynchronous execution fundamentally alters action semantics for world-model controllers, causing planning models to hallucinate future actions and recurrent models to misattribute transitions.
World-model controllers rely on action-conditioned dynamics for prediction and planning, yet real control systems often execute commands asynchronously due to communication delay, packet loss, reordering, and actuator buffering. We study how asynchronous execution changes the action semantics assumed within world-model controllers, rather than treating it only as an external control disturbance. Through controlled interventions, we identify two architecture-dependent failure modes: planning-based controllers such as TD-MPC2 suffer from a future-action timeline mismatch between imagined and executed action sequences, while recurrent world models such as DreamerV3 can attribute observed transitions to commands that were not actually applied. Our analysis shows that TD-MPC2 requires the correct future action sequence during latent dynamics rollout, whereas DreamerV3 requires timely attribution of each transition to the action that generated it. Based on these findings, we introduce two lightweight execution-consistent interfaces, Future-Sequence for TD-MPC2 and Applied-Action Feedback for DreamerV3, that correct these mismatches without modifying the pretrained world models. Experiments across delays, packet loss, reordering, multiple control domains, measured network traces, and a process-separated asynchronous stack consistently support both diagnoses and the corresponding architecture-specific corrections.