Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces VERAGMIL, a novel framework that integrates a high-fidelity simulator with a user-friendly Virtual Reality (VR) interface to facilitate the collection of human demonstrations for training Robot-Assisted Feeding (RAF) systems. The research highlights the challenges of manipulating granular food items and demonstrates that VR-based demonstrations significantly enhance the performance of imitation learning models compared to traditional 3D space mouse data. Notably, the BCQ model achieved the best results, particularly in minimizing spillage and closely matching human performance in scooping and transporting tasks.
VR-based demonstrations can dramatically improve robot performance in handling granular foods, outperforming traditional methods and nearing human capabilities.
Robot-Assisted Feeding (RAF) systems are essential for assisting individuals with disabilities or motor impairments in eating tasks. Manipulating granular food items, such as rice and beans, poses significant challenges due to their dynamic physical properties. Learning from human demonstrations offers a promising solution, but acquiring high-quality demonstrations is complex. To address this, we present VERAGMIL, a framework that combines a high-fidelity simulator with an intuitive Virtual Reality (VR) interface for recording demonstrations and supporting different imitation learning methods. VERAGMIL provides a realistic environment for training RAF systems to handle granular materials, including robots, sensors, and various food items with distinct physical characteristics. We evaluate VERAGMIL by training three imitation learning models, BC, BC-RNN, and BCQ, on granular scooping and transporting tasks using both VR interface and 3D space mouse demonstrations, comparing them with a human-expert baseline. The models are assessed on success rate, spillage, generalization to unseen food items, and task completion time. Results show that VR-based demonstrations significantly outperform 3D space mouse data, with BCQ achieving the best overall performance, particularly in reducing spillage and approaching human performance. These findings underscore the effectiveness of our framework for training RAF systems in granular material handling. The code for our framework is publicly available at: https://github.com/AmanuelErgogo/VERAGMIL.git.