Search papers, labs, and topics across Lattice.
This paper introduces perceptive shared autonomy, a framework that modulates autonomy levels based on uncertainty estimates from deep learning-based perception to improve robustness in robotic manipulation. They use Neural Tangent Kernel (NTK)-based point cloud registration to quantify uncertainty in perception. A user study with 15 participants on aerial manipulation tasks demonstrates that SPIRIT, the resulting system, enhances both manipulation performance and reliability by switching between semi-autonomous and haptic teleoperation modes based on the perception uncertainty.
Robots can nimbly switch between autonomous and teleoperated modes based on the confidence of their learned perception, leading to more reliable manipulation.
Deep learning (DL) has enabled impressive advances in robotic perception, yet its limited robustness and lack of interpretability hinder reliable deployment in safety critical applications. We propose a concept termed perceptive shared autonomy, in which uncertainty estimates from DL based perception are used to regulate the level of autonomy. Specifically, when the robot's perception is confident, semi-autonomous manipulation is enabled to improve performance; when uncertainty increases, control transitions to haptic teleoperation for maintaining robustness. In this way, high-performing but uninterpretable DL methods can be integrated safely into robotic systems. A key technical enabler is an uncertainty aware DL based point cloud registration approach based on the so called Neural Tangent Kernels (NTK). We evaluate perceptive shared autonomy on challenging aerial manipulation tasks through a user study of 15 participants and realization of mock-up industrial scenarios, demonstrating reliable robotic manipulation despite failures in DL based perception. The resulting system, named SPIRIT, improves both manipulation performance and system reliability. SPIRIT was selected as a finalist of a major industrial innovation award.
