Search papers, labs, and topics across Lattice.
The paper introduces SaiVLA-0, a novel Vision-Language-Action architecture inspired by the brain's cerebrum, pons, and cerebellum, designed for compute-aware robotic control. The architecture uses a frozen "Cerebrum" for high-level multimodal priors, a "Pons Adapter" to integrate these priors with proprioception, and a "Cerebellum" (ParaCAT) for fast, parallel categorical decoding. Preliminary LIBERO experiments demonstrate that split feature caching reduces training time and improves success rates, with SaiVLA0 achieving 99.0% mean success.
Mimicking the brain's architecture unlocks faster training (40% reduction) and higher success rates (6% improvement) in vision-language-action tasks.
We revisit Vision-Language-Action through a neuroscience-inspired triad. Biologically, the Cerebrum provides stable high-level multimodal priors and remains frozen; the Pons Adapter integrates these cortical features with real-time proprioceptive inputs and compiles intent into execution-ready tokens; and the Cerebellum (ParaCAT) performs fast, parallel categorical decoding for online control, with hysteresis/EMA/temperature/entropy for stability. A fixed-ratio schedule and two-stage feature caching make the system compute-aware and reproducible. Inspired by active, foveated vision, our wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view. The design is modular: upgrading the Cerebrum only retrains the Pons; changing robots only trains the Cerebellum; cerebellum-only RL can further refine control without touching high-level semantics. As a concept-and-protocol paper with preliminary evidence, we outline a timing protocol under matched conditions (GPU, resolution, batch) to verify anticipated efficiency gains. We also report preliminary LIBERO evidence showing that split feature caching reduces training time (7.5h to 4.5h) and improves average success (86.5% to 92.5%) under official N1.5 head-only training, and that SaiVLA0 reaches 99.0% mean success.
