The Effects of VR and TP Visual Cues on Motor Imagery Subjects and Performance

Abstract

This study objectively evaluated the effects of Virtual Reality Visual Cues (VRVCs) and Traditional Plane Visual Cues (TPVCs) on motor imagery (MI) subjects and Brain-Computer Interface (BCI) performance when building a classification model for MI-BCIs. Four metrics, namely, imagery stability, brain activation and connectivity, classification accuracy, and fatigue level, were used to evaluate the effects of TPVCs and VRVCs on subjects and MI-BCI performance. Nine male subjects performed four types of MI (left/right-hand grip strength) under VRVCs and TPVCs while EEG and fNIRS signals were acquired. FBCSP and HFD were used to extract features, and KNN was used to evaluate MI-BCI accuracy. Rt-DTW was used to evaluate MI stability. PSD topography and the brain functional network were used to assess brain activation and connectivity. Cognitive load and fNIRS mean features were used to evaluate fatigue. The mean classification accuracies of the four types of MI under TPVCs and VRVCs were 50.83% and 51.32%, respectively. However, MI was more stable under TPVCs. VRVCs enhanced the connectivity of the brain functional network during MI and increased the subjects’ fatigue level. This study’s head-mounted VRVCs increased the subjects’ cognitive load and fatigue level. By comparing the performance of an MI-BCI under VRVCs and TPVCs using multiple metrics, this study provides insights for the future integration of MI-BCIs with VR.