Search papers, labs, and topics across Lattice.
This paper introduces G$^3$VLA, a novel geometric module that enhances vision-language-action (VLA) models by integrating calibrated geometric information into the visual-token stream. By employing intrinsic-conditioned ray embeddings, projective positional encoding, and bidirectional cross-view fusion, G$^3$VLA addresses the mismatch between 2D image coordinates and the robot's geometric context, particularly in multi-camera setups. The results demonstrate significant performance improvements across various benchmarks, especially in tasks requiring spatial awareness and object sensitivity, without altering the model's action space or imitation objectives.
G$^3$VLA shows that incorporating calibrated geometric structure into VLA models can lead to substantial performance gains in robot manipulation tasks.
Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras -- a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G$^3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated $蟺^3$X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on $蟺_0$, G$^3$VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on $蟺_{0.5}$ and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at https://sites.google.com/view/g3vla
