Search papers, labs, and topics across Lattice.
This paper introduces WAM-Diff2, a hierarchical framework that transforms pre-trained autoregressive Vision-Language-Action (VLA) models into efficient parallel diffusion models for autonomous driving. By employing a three-stage hierarchical distillation strategy, the method addresses the architectural challenges of mismatched attention patterns and divergent optimization objectives, maintaining semantic integrity while enhancing execution speed. The results show that WAM-Diff2 not only achieves performance parity with traditional autoregressive models but also provides a significant 2.8x speedup in decoding, scaling up to 15.1x with additional optimizations.
Transforming autoregressive VLA models into diffusion frameworks yields a 2.8x decoding speedup while preserving performance parity, revolutionizing efficiency in autonomous driving.
Vision-Language-Action (VLA) models have emerged as a prominent paradigm for end-to-end autonomous driving; however, their efficient deployment is severely constrained by high computational latency and exposure bias arising from sequential autoregressive decoding. Conversely, while specialized diffusion policies enable low-latency, parallel execution, training them from scratch typically yields narrow, single-task architectures that lack holistic visual-linguistic reasoning. Successfully transforming pre-trained autoregressive generalists into parallel diffusion models could combine multi-task cognitive intelligence with execution efficiency, yet this transition presents a formidable architectural challenge due to mismatched attention patterns (causal versus bidirectional) and divergent optimization objectives. To bridge this divide, we introduce WAM-Diff2, a multi-task discrete diffusion VLA framework powered by a three-stage hierarchical distillation strategy. By structuring the architectural shift through progressive block-wise adaptation, block-wise distillation, and model-wise cross-scale distillation, WAM-Diff2 preserves the underlying semantic foundations of the base model while accelerating inference. Extensive evaluations across driving understanding, perception, and planning benchmarks demonstrate that WAM-Diff2 effectively mitigates exposure bias and achieves performance parity with autoregressive baselines. Crucially, the autoregressive-to-diffusion transition yields a 2.8x decoding speedup, which scales to an ultimate 15.1x acceleration when combined with system-level optimizations including FlashInfer and CUDA Graphs.
