Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an augmented reality brain-robot interface (AR BRI) that facilitates generalist robot arm manipulation through a combination of gaze-based object selection and motor imagery action control. By integrating eye-tracking technology and context-aware visual overlays, the system enhances user interaction and task execution in real-world settings, as evidenced by a feasibility study involving 18 participants performing daily activities. The results indicate effective sequential task execution and high user engagement, achieving a usability rating above 70, thus validating the potential of this paradigm for complex robotic assistance applications.
Eye-tracking combined with augmented reality transforms how users interact with robots, enabling intuitive control for complex tasks like cooking and cleaning.
The integration of augmented reality (AR) and EEG-based brain-computer interfaces (BCIs) offers a promising path for enabling intuitive control of robots for assistive purposes. However, existing AR brain-robot interface (BRI) systems are often constrained to task-specific structures, limiting their utility in real-world environments. We present an AR BRI designed for generalist robot arm manipulation that combines gaze-based object selection with motor imagery action control. Our system uses eye-tracking for intuitive object targeting and context-aware visual overlays ("Place" and "Use") to guide the user through tasks within a shared autonomy framework. We evaluated the interface through a feasibility study with 18 healthy participants performing three multi-step activities of daily living: drinking, using a drawer, and operating an oven. Our results demonstrate that this interaction paradigm enables effective sequential task execution and high user engagement, achieving a "Good" usability rating (SUS > 70). These findings support the feasibility of the proposed interaction paradigm for complex BCI-driven robotic assistance, and motivate future evaluation with the intended target population. Project website: https://ar-bri-manip.github.io/.