Search papers, labs, and topics across Lattice.
This paper introduces fARfetch, an augmented reality-human-robot collaboration (AR-HRC) system designed to enhance interaction in large, visually diverse environments. By integrating shared semantic mapping, a context-aware miniature representation, and vision-language model-driven content adaptation, fARfetch significantly improves task performance and reduces user workload. The results from a user study demonstrate that fARfetch allows for 66% faster task completion and substantial reductions in mental and temporal demand, as well as frustration levels.
fARfetch accelerates outdoor robotic inspections by 66% while slashing user frustration by two-thirds through innovative AR content adaptation.
Augmented Reality (AR) can improve collocated human-robot collaboration by making robot state and intent visible and enabling intuitive control, yet large, visually diverse environments like the outdoors challenge both interaction and content legibility, especially at long distances and beyond visual line of sight. We present fARfetch, an AR-HRC system that integrates (i) shared semantic environment mapping across an AR headset and robot that visualizes detected landmarks in AR to support landmark-grounded go-to commands, (ii) a context-aware world-in-miniature representation of the shared environment for fine-grained path authoring, and (iii) vision-language-model driven AR view management that jointly adapts virtual content color, size, and orientation to maintain legibility in large visually diverse environments. We implement fARfetch with a Meta Quest 3 headset and Unitree Go2 quadruped robot, and conduct a within-subjects user study (N=13) on a real-world large-scale (30.5m) outdoor inspection task. fARfetch yielded significantly faster completion times than a non-AR baseline (66%) and significantly lower workload in mental demand (-43%), temporal demand (-34%), and frustration (-66%). A custom legibility survey indicated fARfetch effectively maintained virtual content legibility in the large outdoor environment.
