Search papers, labs, and topics across Lattice.
This paper introduces an augmented reality (AR) interface using Microsoft HoloLens 2 for teleoperating soft robots, specifically addressing the challenge of modeling these robots accurately and efficiently. The system estimates the robot's position using sensor data and physics-based simulation within the AR environment. Experiments on a pneumatic manipulator ("PETER") showed position estimation errors around 5% of the robot's length, indicating the feasibility of AR-based teleoperation for soft robots.
AR teleoperation can now handle the squishy complexity of soft robots, achieving ~5% accuracy in position estimation using a physics-based model.
Although virtual and augmented reality are gaining traction as teleoperation tools for various types of robots, including manipulators and mobile robots, they are not being used for soft robots. The inherent difficulties of modelling soft robots mean that combining accurate and computationally efficient representations is very challenging. This paper presents an augmented reality interface for teleoperating these devices. The developed system consists of Microsoft HoloLens 2 glasses and a central computer responsible for calculations. Validation is performed on PETER, a highly modular pneumatic manipulator. Using data collected from sensors, the computer estimates the robot's position based on the physics of the virtual reality programme. Errors obtained are on the order of 5% of the robot's length, demonstrating that augmented reality facilitates operator interaction with soft manipulators and can be integrated into the control loop.
