Search papers, labs, and topics across Lattice.
This paper introduces Deltoris, a novel algorithm-hardware co-design framework that enhances the efficiency of diffusion-based vision-language-action (VLA) models for real-time inference in embodied AI. By leveraging temporal-aware bit-sparsity to compute only the differences between consecutive inputs and employing speculative inference to reduce off-chip traffic, Deltoris significantly alleviates the computational burden on edge devices. The results demonstrate that Deltoris achieves up to 34.2脳 speedup over mobile GPUs and 6.1脳 over existing accelerators, all while preserving comparable accuracy levels.
Deltoris achieves a staggering 34.2脳 speedup for real-time VLA inference, revolutionizing how embodied AI can operate on edge devices.
Vision-language-action (VLA) models have emerged as a key component in embodied AI. Among existing approaches, diffusion-based VLA models achieve superior motion quality and generalization. However, diffusion-based VLA models are compute-intensive and must run at high control frequency, e.g., 50-200 Hz. Thus, it imposes strict latency and energy constraints on edge devices. In this work, we present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference. First, we exploit the temporal similarity of consecutive inputs and propose a \textit{temporal-aware bit-sparsity} algorithm that computes only the differences between consecutive inputs, eliminating redundant bit-level operations. To further address the extra off-chip traffic introduced by our algorithm, we propose a \textit{speculative inference} technique, which amortizes data loading across multiple control steps. Lastly, to support these techniques, we co-design a dedicated accelerator with customized 1D systolic bit-serial PE arrays that eliminate PE workload imbalance. Our evaluation shows that Deltoris achieves up to 34.2$\times$ speedup over mobile GPUs and 6.1$\times$ over prior accelerators, while maintaining comparable accuracy.
