Search papers, labs, and topics across Lattice.
This paper introduces an augmented reality (AR)-based physical human-robot interaction (pHRI) framework for teleoperating hybrid rigid-soft robots, enabling intuitive grasping in unstructured environments. The system allows users to interact with a simulated robot model within a physics engine, superimposed on the real robot via an AR headset, facilitating simulated execution before real-world deployment. A key component is a real-to-simulation parameter identification pipeline that leverages the soft robot's geometric properties to accurately model its static and dynamic behavior, ensuring consistency between virtual and physical robots.
Control hybrid rigid-soft robots with the ease of AR teleoperation, thanks to a new pipeline that accurately models the soft robot's real-world behavior in simulation.
Hybrid rigid-soft robots combine the precision of rigid manipulators with the compliance and adaptability of soft arms, offering a promising approach for versatile grasping in unstructured environments. However, coordinating hybrid robots remains challenging, due to difficulties in modeling, perception, and cross-domain kinematics. In this work, we present a novel augmented reality (AR)-based physical human-robot interaction framework that enables direct teleoperation of a hybrid rigid-soft robot for simple reaching and grasping tasks. Using an AR headset, users can interact with a simulated model of the robotic system integrated into a general-purpose physics engine, which is superimposed on the real system, allowing simulated execution prior to real-world deployment. To ensure consistent behavior between the virtual and physical robots, we introduce a real-to-simulation parameter identification pipeline that leverages the inherent geometric properties of the soft robot, enabling accurate modeling of its static and dynamic behavior as well as the control system's response.
MIT CSAIL
NUS