Search papers, labs, and topics across Lattice.
This paper introduces GS-VLA, a novel framework that enhances the robustness of frozen Vision-Language-Action (VLA) policies to viewpoint shifts through Gaussian-based novel-view synthesis. By addressing the limitations of existing methods that require extensive retraining or risk catastrophic forgetting, GS-VLA effectively normalizes viewpoint discrepancies without altering policy weights. Experimental results demonstrate significant performance recovery on the LIBERO benchmark, with improvements across various policy architectures, unseen tasks, and perturbation scales, highlighting the framework's versatility and efficiency.
A lightweight Gaussian-based module can restore up to 80% of performance lost due to viewpoint shifts in VLA policies without any retraining.
This paper proposes a lightweight, plug-and-play framework that improves robustness to viewpoint shifts in Vision-Language-Action (VLA) policies without policy retraining. To our knowledge, this is the first approach to directly leverage 3D Gaussian-based novel-view synthesis for observation-space adaptation in VLA policies. Current VLA performance relies on the implicit assumption that training and deployment camera configurations are identical. Our experiments show that even a small displacement of the camera mount can reduce the success rate on the LIBERO benchmark from about 90% to about 10% in the worst case. Prior approaches, such as large-scale fine-tuning or generative data augmentation, are computationally expensive and risk catastrophic forgetting. To address this, viewpoint shifts are reformulated as a localized novel-view synthesis problem. Under a Locality assumption, that camera perturbations remain within a small bounded region relative to the workspace, viewpoint normalization reduces to a scene- and policy-independent disocclusion task. Our work implements this idea with a 4M-parameter 3D-Gaussian canonicalizer prepended to a frozen VLA policy. Without modifying policy weights, GS-VLA improves performance across three orthogonal axes: (1) Policy architectures, (2) Unseen task suites, and (3) Perturbation scales. These results show that a lightweight visual module can recover a large fraction of the performance lost under viewpoint shift, without policy retraining.
