Search papers, labs, and topics across Lattice.
Assistron is a shared autonomy model that integrates Vision-Language-Action (VLA) models to assist users in daily tasks by minimizing cognitive and physical effort. It intelligently executes macro-movements based on verbal commands while allowing for human intervention during critical failure points, thus maintaining the model's broad behavioral capabilities without fine-tuning. Empirical validation shows that Assistron enhances task success rates and reduces user workload compared to traditional teleoperation methods, marking a significant advancement in assistive manipulation technologies.
Assistron achieves a remarkable balance between autonomy and user control, significantly enhancing task success while reducing user effort in daily activities.
We propose Assistron, a shared autonomy model that leverages Vision-Language-Action (VLA) models to assist the user in daily activities. Our approach is grounded in two core principles: (1)~minimizing human cognitive and physical effort by leveraging VLA-driven autonomy for macro-movements, and (2)~prioritizing human intervention specifically at critical failure points. Driven by the user's verbal language commands, Assistron utilizes the VLA to autonomously execute macro-reaching trajectories, saving users' effort. In contact-rich interactions where VLAs tend to fail, Assistron employs a phase-aware interaction detection mechanism and solicits the user to intervene, in turn adjusting the VLA's action generation via flow matching guidance. Critically, our formulation eliminates the need for VLA fine-tuning, protecting its broad behavioral priors from catastrophic forgetting and ensuring the model does not become a narrow specialist. We validate our approach on a comprehensive multi-task scene recovery benchmark encompassing diverse daily manipulation skills. Empirical results demonstrate that Assistron significantly improves task success rates over pure autonomous baselines while significantly reducing human cognitive and physical workload compared to traditional teleoperation, offering a scalable, smooth, and effortless paradigm for assistive manipulation. The code is available in https://github.com/mousecpn/Assistron.git.
