Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates the accuracy of hand pose estimation in augmented reality (AR) applications, specifically using the HoloLens 2 and state-of-the-art algorithms, in individuals with cervical spinal cord injury (cSCI) compared to uninjured controls. The findings reveal that pose estimation accuracy is comparable across both groups, indicating that the HoloLens 2 and advanced algorithms can effectively generalize to those with hand impairments. Additionally, the research highlights a slight accuracy advantage when interacting with clear objects over opaque ones, suggesting practical implications for AR-based hand rehabilitation tools.
Pose estimation accuracy in AR applications remains robust even for individuals with significant hand impairments, challenging assumptions about technology limitations in rehabilitation contexts.
Mixed reality applications can be designed for hand rehabilitation. Augmented reality (AR) head mounted displays (HMDs) specifically allow for ecologically valid tasks because individuals can see their real environment and interact with real objects while receiving additional cues on the HMD. While these applications rely on accurate hand pose estimation, there is a gap in investigating the influence of hand impairment or occlusion from real-object interactions on pose estimation accuracy. Further, comparisons between AR HMD predictions and state-of-the-art pose estimation methods have not been established. The current study assessed pose estimation accuracy of the HoloLens 2 HMD and state-of-the-art pose estimation algorithms (WiLoR, HaMeR, WildHands, and MediaPipe) while individuals with cervical spinal cord injury (cSCI; n = 13, Neurological Level of Injury: C3-C6; American Spinal Injury Association Impairment Scale: A-D) and 15 uninjured controls interacted with clear and opaque objects. Ground truth estimates of 3D joint positions were generated via triangulation from a multi-camera setup. Pose estimation accuracy did not differ between the cSCI and uninjured control groups suggesting that 3D joint predictions from the HoloLens 2 and pose estimation algorithms can generalize to populations with hand impairment. Further, clear objects provided a small accuracy advantage over opaque objects (0.1 mm) and predictions from both WiLoR and HaMeR were slightly more accurate than the HoloLens 2 (2 mm). Overall, these results suggest that the HoloLens 2 may be viable for hand rehabilitation applications and the dataset generated can be used to refine pose estimation methods for hand-impaired populations.