Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Shared Autonomy for Policy Steering (SAPS), a novel framework that integrates real-time human teleoperation with pretrained Vision-Language-Action (VLA) policies without requiring policy retraining or auxiliary models. By employing three arbitration strategies, including a dynamic Cosine-similarity approach, SAPS significantly enhances task success rates—up to 82%—in both simulated and real-world environments while reducing the need for human intervention. The findings underscore the effectiveness of action-level shared autonomy in improving robot manipulation tasks, making it a viable solution for assistive teleoperation and efficient data collection.
Blending human teleoperation with pretrained policies can boost robot task success rates by up to 82% while minimizing cognitive load on operators.
Recent advancements in Vision-Language-Action (VLA) models have demonstrated impressive generalist capabilities in robot manipulation, yet these policies can be brittle under out-of-distribution spatial and semantic perturbations. While human teleoperation offers reliable recovery, it can demand high cognitive load and precise manual control, and existing policy steering methods often require auxiliary models or sampler modifications. In this work, we introduce Shared Autonomy for Policy Steering (SAPS), a framework that blends real-time human teleoperation commands with pretrained policy actions at the action level. SAPS requires no policy retraining, auxiliary dynamics models, or architectural modifications. We propose and evaluate three arbitration strategies to balance human and VLA policy control, including a dynamic Cosine-similarity arbitration strategy that computes the geometric agreement between human and policy actions. Across evaluations in simulation (LIBERO, LIBERO-PRO, CALVIN) and on real-world robot hardware, SAPS improves task success rates over autonomous execution by up to 82% in both simulation and the real world. Furthermore, our approach drastically reduces human intervention compared to pure teleoperation, while simultaneously achieving faster task completion times than both autonomous execution and pure teleoperation. These results demonstrate that action-level shared autonomy is a practical, model-agnostic approach for reliably deploying generalist robot policies in real-world contexts involving a human operator,with promising applications in assistive teleoperation and scalable data collection.