Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenges of robotic manipulation of textiles by developing a synthetic data generation pipeline using Blender to create auto-annotated keypoints, which are combined with real-world data to train a wrinkle detection model. The authors introduce a perception framework that integrates a convolutional neural network (CNN) for detecting permutation-invariant keypoints and a YOLOv8-OpenCV pipeline for extracting grasping points from wrinkles. The resulting bimanual algorithm demonstrates effective manipulation of fully folded garments, achieving a Mean Position Error (MPE) of 1.7615 pixels and successfully transferring to physical fabrics without the need for fine-tuning, outperforming existing methods in high-occlusion scenarios.
A synthetic data pipeline enables robots to manipulate textiles with unprecedented accuracy, achieving a remarkable MPE of just 1.7615 pixels in real-world applications.
Robotic manipulation of textiles remains challenging because continuous deformation and self-occlusions hinder the robust visual perception required to estimate the cloth's state. To address the lack of annotated real-world data, we developed a Blender-based synthetic pipeline exporting auto-annotated keypoints, and combined manually labeled renders with real-world data to train a wrinkle detector. We present a perception framework integrating a CNN for permutation-invariant keypoint detection and a YOLOv8-OpenCV pipeline to extract grasping points from structural wrinkles. A proposed bimanual algorithm uses this system to stretch fully folded garments via wrinkles, transitioning to keypoint-based ironing once corners emerge. The keypoint model achieves a Mean Position Error (MPE) of 1.7615 pixels. The perception system transfers to physical fabrics without fine-tuning, outperforming baselines that fail in high-occlusion states or yield false positives on severe folds.