Search papers, labs, and topics across Lattice.
This paper introduces SpecVLA, a novel algorithm-architecture co-design framework that enhances the efficiency of Vision-Language-Action (VLA) models for real-time robotic manipulation. By leveraging the alternating active and inactive states of robotic environments, SpecVLA allows for long-action-length speculative predictions during inactive phases and selective verification during active phases. Evaluations demonstrate that this approach significantly reduces end-to-end latency while maintaining high task success rates across multiple benchmarks, marking a substantial advancement in embodied AI deployment.
SpecVLA achieves real-time robotic manipulation by enabling long-action-length predictions with timely verification, cutting down latency without sacrificing reliability.
Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment. Although Dadu-Corki, a dedicated accelerator for efficient embodied AI, has been introduced, it does not exploit the inherent interaction patterns between the robot and its environment, which results in a relatively short predicted action length. We observe that robotic environments naturally alternate between active states-where precise actions are crucial-and inactive states-where actions have limited impact on task success. This insight enables a new scheduling opportunity: long-action-length speculative prediction in inactive states, paired with selective verification in active states. We propose SpecVLA, an algorithm-system co-design framework that adaptively balances action length, inference latency, and task reliability. On the algorithm side, SpecVLA introduces a state-aware VLA inference execution paradigm and a hardware-friendly construction of a smaller verification model (sVLA) using differential residuals and block-wise mixed-precision quantization. On the system side, we develop a heterogeneous architecture consisting of a GPU and a robotic-specific hardware module, along with a speculative dataflow that decouples VLA and sVLA through parallel execution. Comprehensive evaluations on OpenVLA and RDT across LIBERO and ManiSkill benchmarks show that SpecVLA reduces end-to-end latency significantly while preserving task success rate. By enabling long-action-length speculative prediction with timely verification, SpecVLA achieves real-time robotic manipulation with both high efficiency and reliability.
