Search papers, labs, and topics across Lattice.
The study introduces CT-VAM, a cerebello-thalamic-inspired model designed for efficient visuomotor control by predicting action chunks from dual-view visual observations and proprioception. This model effectively separates high-level semantic reasoning from low-level execution, allowing for a cloud-edge paradigm where large models handle complex reasoning while CT-VAM executes tasks locally with only 68M parameters. Key results show that CT-VAM achieves competitive LIBERO success rates and reduced inference latency compared to larger vision-language-action models, demonstrating its practicality for resource-constrained robotic platforms.
CT-VAM achieves competitive performance with a fraction of the parameters, enabling efficient real-time control for robotic manipulation.
Vision-language-action models have shown strong promise for robot manipulation, yet raw language is primarily needed to specify task intent rather than to be repeatedly processed during high-frequency low-level execution. Motivated by this separation, we propose a cerebello-thalamic-inspired vision-action model (CT-VAM) for efficient task-conditioned visuomotor control. CT-VAM acts as a compact local execution policy that predicts action chunks from dualview visual observations, proprioception, and a lightweight task condition, potentially enabling a practical cloud-edge paradigm in which high-level semantic reasoning can be handled by large models while fast closed-loop control runs on local hardware. To fuse heterogeneous inputs effectively, CT-VAM introduces TARS (Thalamic Action Routing Stream), a stream-separated conditional attention decoder that independently routes action, visual and task streams, preventing dense sensory tokens from overwhelming compact task-relevant conditions. With only 68M parameters, CT-VAM achieves LIBERO success rates competitive with substantially larger VLA models, while reducing inference latency. Together with flow-consistent inpainting for asynchronous chunk execution, CT-VAM supports high-frequency control and demonstrates robust realworld deployment on resource-constrained robotic platforms.
