Search papers, labs, and topics across Lattice.
This study explores the transferability of Vision-Language-Action (VLA) models from controlled environments to a real-world robotic platform, specifically the UR5e manipulator, through a comprehensive data acquisition and fine-tuning process. The research identifies critical factors influencing the performance gap between offline metrics and real-world execution, emphasizing the importance of action semantics, coordinate frame conventions, and dataset quality. Key findings suggest that successful deployment of VLA systems hinges more on the integrity of the data-model-control pipeline than on merely enhancing model capacity.
Real-world deployment of VLA systems reveals that the success lies in the meticulous management of the entire data-model-control pipeline rather than just model improvements.
This project investigates whether recent Vision-Language-Action (VLA) models can be transferred from controlled research benchmarks to a real-world robotic platform, specifically a UR5e manipulator, in a reproducible and operationally meaningful manner. The work integrates real-robot data acquisition, dataset engineering (compatible with the RLDS format), and the fine-tuning and deployment of OpenVLA and OpenVLA-OFT models, with systematic validation of action representations and control interfaces. The project resulted in several foundational assets: (i) a complete real-robot data acquisition pipeline, (ii) a dataset conversion workflow aligned with RLDS standards, (iii) an initial fine-tuning and inference infrastructure for VLA models, and (iv) a structured set of experimental observations grounded in real-robot trials. These elements collectively establish a reproducible framework for evaluating learning-based manipulation systems beyond simulation. Empirically, the experiments reveal a consistent gap between promising offline indicators and unstable closed-loop behavior on the physical system: this gap cannot be attributed solely to model limitations, it is strongly influenced by action semantics, coordinate frame conventions, temporal alignment between modalities, image preprocessing consistency, and dataset coverage and quality. These observations lead to a key interpretation: the successful deployment of VLA systems in real-world settings depends less on incremental improvements in model capacity and more on precise control of the entire data-model-control pipeline. The project reframes VLA-based robotics from a primarily model-centric challenge to a system-level problem; it highlights the difficulty of running robust task execution on the real robot and provides a clear, experimentally grounded understanding of the conditions required for reliable deployment.
