Search papers, labs, and topics across Lattice.
This paper introduces Drive-HWM, a hierarchical world modeling framework that separates future representation prediction and action generation into slow and fast models, respectively, to enhance autonomous driving capabilities. By employing Dynamic-Aware Latents derived from optical-flow prediction, the slow model captures long-term scene evolution while the fast model predicts immediate actions and next frames based on the latest observations. Experimental results on NAVSIM v1 and v2 show that Drive-HWM significantly improves driving performance, validating the effectiveness of its hierarchical design and joint prediction approach.
A novel hierarchical framework enables autonomous vehicles to anticipate long-term traffic dynamics while making real-time decisions, achieving superior driving performance.
World models offer a promising paradigm for autonomous driving by predicting how traffic scenes may evolve and using such predictions to support action generation. However, existing approaches either separate future prediction from action generation or jointly predict them at the same temporal scale, making it difficult to simultaneously achieve long-horizon anticipation and responsive, observation-grounded decision making. We present Drive-HWM, a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales. The slow world model predicts multi-step future representations to capture extended scene evolution. To explicitly model the abundant motion dynamics in driving environments, we introduce Dynamic-Aware Latents learned through optical-flow prediction. Guided by these future representations, the fast model uses a lightweight multimodal backbone and an autoregressive expert to jointly predict the next frame and the immediate action from the latest observation. Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive. Extensive experiments on NAVSIM v1 and v2 demonstrate the strong driving performance of Drive-HWM. Comprehensive ablation studies further validate the effectiveness of the hierarchical slow--fast design, dynamics-aware future representations, and joint next-frame and action prediction.
