Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
This paper introduces Deltoris, a novel algorithm-hardware co-design framework that enhances the efficiency of diffusion-based vision-language-action (VLA) models for real-time inference in embodied AI. By leveraging temporal-aware bit-sparsity to compute only the differences between consecutive inputs and employing speculative inference to reduce off-chip traffic, Deltoris significantly alleviates the computational burden on edge devices. The results demonstrate that Deltoris achieves up to 34.2× speedup over mobile GPUs and 6.1× over existing accelerators, all while preserving comparable accuracy levels.
Deltoris achieves a staggering 34.2× speedup for real-time VLA inference, revolutionizing how embodied AI can operate on edge devices.
Vision-language-action (VLA) models have emerged as a key component in embodied AI. Among existing approaches, diffusion-based VLA models achieve superior motion quality and generalization. However, diffusion-based VLA models are compute-intensive and must run at high control frequency, e.g., 50-200 Hz. Thus, it imposes strict latency and energy constraints on edge devices. In this work, we present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference. First, we exploit the temporal similarity of consecutive inputs and propose a \textit{temporal-aware bit-sparsity} algorithm that computes only the differences between consecutive inputs, eliminating redundant bit-level operations. To further address the extra off-chip traffic introduced by our algorithm, we propose a \textit{speculative inference} technique, which amortizes data loading across multiple control steps. Lastly, to support these techniques, we co-design a dedicated accelerator with customized 1D systolic bit-serial PE arrays that eliminate PE workload imbalance. Our evaluation shows that Deltoris achieves up to 34.2$\times$ speedup over mobile GPUs and 6.1$\times$ over prior accelerators, while maintaining comparable accuracy.