Search papers, labs, and topics across Lattice.
This paper introduces a digital twin framework for visuo-haptic teleoperation of complex-shaped, optically-driven microrobots, addressing the need for improved operator awareness in delicate biomedical tasks. The framework combines image-based pose estimation, microrobot motion simulation using a Multi-Sphere Distributed Manipulation (MSDM) model, and model-based haptic rendering within a ROS-connected bimanual teleoperation system. Results from simulated cell-delivery tasks demonstrate that haptic feedback significantly improves task success and reduces variability in contact force and microrobot positioning.
Haptic feedback in optical microrobotics isn't just a nice-to-have; it can boost cell delivery success rates by 50% in simulation.
Optical tweezers (OT) provide piconewton-scale manipulation for delicate biomedical tasks, where visuo-haptic feedback can improve operator awareness by conveying interaction-force cues and trap-stability information. However, visuo-haptic teleoperation frameworks for complex-shaped optical microrobots remain underdeveloped, particularly in multi-trap manipulation scenarios. This paper presents a digital twin framework for virtual visuo-haptic teleoperation of complex-shaped OT-driven microrobots. The framework integrates a digital twin environment, image-based pose and depth estimation, microrobot motion simulation, and model-based haptic rendering within a Robot Operating System (ROS)-connected bimanual teleoperation system. For force modeling, we combine a Multi-Sphere Distributed Manipulation (MSDM) model with optical-force estimation from the Optical Tweezers Toolbox, enabling simulator-driven visuo-haptic feedback. The framework reproduces representative microrobot motion trends and provides haptic force rendering that is numerically consistent with the fitted optical-force model. In simulated cell-delivery tasks, haptic feedback reduced the standard deviations of the contact-force metric and the microrobot-to-trap-center distance metric by 53.2% and 55.2%, respectively, and improved task success from 30% to 80%. These results demonstrate the framework's effectiveness for evaluating visuo-haptic teleoperation strategies for complex-shaped optical microrobots.
