Search papers, labs, and topics across Lattice.
StellaVLA introduces a novel framework for Vision-Language-Action (VLA) models that enables them to adapt at test time using a single retrieved structured demonstration, rather than requiring extensive data collection and fine-tuning. By converting raw trajectories into structured formats that include task plans and verbalized 3D motions, the framework allows models to reason about tasks, enhancing their generalization capabilities across different embodiments. Achieving a top rank on the VLA-Arena leaderboard and demonstrating high success rates in real-robot benchmarks, StellaVLA significantly outperforms prior models, showcasing its effectiveness in out-of-distribution scenarios.
StellaVLA achieves a remarkable 98.8% success rate in adapting VLA models to out-of-distribution tasks using structured demonstrations without human annotation.
Vision-Language-Action (VLA) models can follow instructions and manipulate objects, but their performance often collapses out of distribution (OOD), when the scene, viewpoint, or object differs from training. Adapting to each new situation typically requires collecting more data and fine-tuning. We present StellaVLA, a framework that instead adapts at test time by conditioning on a single retrieved demonstration. The key idea is to move beyond imitating what an expert did and instead convey why: an automated offline pipeline converts each raw trajectory into a structured demonstration, e.g., a task plan, sub-goal descriptions, and verbalized 3D motion, at zero human-annotation cost. Provided as in-context guidance, this structured demonstration lets the policy reason about the task rather than mimic a pixel trajectory, which also makes it transferable across embodiments (real-robot, human-hand, or XR demonstrations). A parallel dual-training design internalizes this reasoning during training through a joint action-and-language objective, while inference uses the action expert alone, preserving real-time, high-frequency control with no added latency. On the VLA-Arena leaderboard(Aug 1, 2026), StellaVLA ranks first with an overall score of 0.63, versus 0.44 and 0.22 for the strong prior models ($\pi_{0.5}$ and LingBot-VLA), and it further leads on LIBERO with 98.8% average success rate and LIBERO-Plus with 85.1% success rate. Our real-robot benchmark demonstrates that StellaVLA can use both human/robot demos and human-to-robot (XR) demos as in-context structured demonstration to help VLA model adapt to OOD tasks.
