Shared Three-Dimensional Robotic Arm Control Based on Asynchronous BCI and Computer Vision

• Main Authors: Yajun Zhou (Author), Tianyou Yu (Author), Wei Gao (Author), Weichen Huang (Author), Zilin Lu (Author), Qiyun Huang (Author), Yuanqing Li (Author)
• Format: Book
• Published: IEEE, 2023-01-01T00:00:00Z.
• Subjects: article
• Online Access: Connect to this object online.
• Notes: 1558-0210; 10.1109/TNSRE.2023.3299350
• Keywords: Asynchronous brain-computer interface (BCI); electroencephalography (EEG); electrooculography (EOG); robotic arm; computer vision; Medical technology; R855-855.5; Therapeutics. Pharmacology; RM1-950
• Source: IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol 31, Pp 3163-3175 (2023)

Objective: A brain-computer interface (BCI) can be used to translate neuronal activity into commands to control external devices. However, using noninvasive BCI to control a robotic arm for movements in three-dimensional (3D) environments and accomplish complicated daily tasks, such as grasping and drinking, remains a challenge. Approach: In this study, a shared robotic arm control system based on hybrid asynchronous BCI and computer vision was presented. The BCI model, which combines steady-state visual evoked potentials (SSVEPs) and blink-related electrooculography (EOG) signals, allows users to freely choose from fifteen commands in an asynchronous mode corresponding to robot actions in a 3D workspace and reach targets with a wide movement range, while computer vision can identify objects and assist a robotic arm in completing more precise tasks, such as grasping a target automatically. Results: Ten subjects participated in the experiments and achieved an average accuracy of more than 92% and a high trajectory efficiency for robot movement. All subjects were able to perform the reach-grasp-drink tasks successfully using the proposed shared control method, with fewer error commands and shorter completion time than with direct BCI control. Significance: Our results demonstrated the feasibility and efficiency of generating practical multidimensional control of an intuitive robotic arm by merging hybrid asynchronous BCI and computer vision-based recognition.