Search papers, labs, and topics across Lattice.
This paper introduces OrthoSkillVLA, a novel framework for continual skill learning in pretrained Vision-Language-Action (VLA) models that addresses catastrophic forgetting by imposing separate subspace constraints on the VLM and ActionHead components. By utilizing a lightweight feature-aware MoE decoder, the method allows for efficient allocation of skill-specific experts while maintaining the integrity of previously learned skills. Extensive evaluations show that OrthoSkillVLA significantly outperforms existing approaches in preserving prior skills during the acquisition of new ones, demonstrating its effectiveness in real-world applications.
OrthoSkillVLA preserves prior skills in pretrained VLA models while seamlessly integrating new ones, outperforming traditional methods in both simulated and real-world scenarios.
Pretrained Vision-Language-Action models provide a strong foundation for robot learning, but sequentially adapting them to diverse skills can perturb the representations and velocity mappings used by previous skills, leading to catastrophic forgetting. Architecture-based approaches improve retention by isolating skills but lead to increased inference footprint. Recent subspace-constrained methods restrict parameter updates in an orthogonal subspace to minimize interference but impose a unified constraint on the entire model. We analyze the distinct roles of internal VLA components and identify two VLA-specific challenges. First, the VLM maintains broad semantic representations, making it vulnerable to capacity exhaustion, whereas the ActionHead refines semantics into localized velocity patterns that are highly sensitive to perturbations. Second, the final velocity decoder serves as a readout layer. Freezing it forms an output-stage expressivity bottleneck, while updating it risks overwriting previous velocity mappings. To this end, we propose OrthoSkillVLA, a parameter-efficient framework for continual skill learning in pretrained VLA models without demonstration replay. Given the representation heterogeneity, we impose separate subspace constraints on the VLM and ActionHead, preserving reusable semantic capacity while protecting localized velocity patterns. For the output layer, we introduce a lightweight feature-aware MoE decoder, where each skill is allocated a compact expert and a training-free router selects the expert according to feature-space affinity. Extensive simulated and real-world evaluations, together with ablations, demonstrate that OrthoSkillVLA better preserves prior skills while acquiring new ones.
