Search papers, labs, and topics across Lattice.
This paper addresses the high inference latency of Vision-Language-Action (VLA) models used in robotic manipulation by identifying and mitigating two key sources of temporal redundancy: repeated visual encoding of similar frames and inefficient multi-step iterative sampling in diffusion-based policies. The authors propose an innovative acceleration strategy that incrementally updates only the tokens for dynamic scene regions and compresses diffusion sampling into a two-step schedule, significantly enhancing computational efficiency. Experimental results on multiple robotic platforms show that this approach achieves over a 2x speedup while maintaining a high success rate of up to 98% on manipulation tasks.
Achieving over 2x speedup in VLA inference without sacrificing performance could revolutionize real-time robotic applications.
Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment. We identify two primary sources of temporal redundancy in existing VLA pipelines: repeated visual encoding of highly similar consecutive frames and multi step iterative sampling in diffusion based policies. To address this, we propose a system level acceleration strategy that reduces computation in both perception and action generation. On the perception side, we incrementally update only tokens corresponding to dynamic scene regions instead of re-encoding entire frames. On the policy side, we compress diffusion sampling into a compact 2-step schedule through efficiency oriented training while preserving action precision. Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance, achieving up to 98% success rate on general manipulation benchmarks. Our codes will be released on Github.
