Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
Fast-VLA addresses the computational inefficiency of traditional VLA policies by invoking a frozen VLM only once per episode to establish a fixed, multi-layer key-value task memory. A lightweight online actor then fuses learned queries with real-time sensory feedback to retrieve context from this memory and predict action chunks, bypassing repeated foundation model inference. This approach achieves state-of-the-art performance on several benchmarks while drastically reducing inference time and memory footprint.
Robot manipulation policies can achieve state-of-the-art performance and 17x faster inference by freezing the VLM and using a lightweight actor to query a fixed task memory.
Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi-layer key-value task memory. During execution, a lightweight online actor fuses learned time- and arm-indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re-running the foundation model. Fast-VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO-Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9-17.1x and requires 1.22 GiB of post-prefill online resident memory. Task-memory swaps further show that the actor relies on episode-matched task memory. These results show that task context can be established once while closed-loop control remains responsive throughout execution. Project page: https://garfield-kh.github.io/FastVLA