Design and Implementation of a Behavioral Sequence Framework for Human–Robot Interaction Utilizing Brain-Computer Interface and Haptic Feedback

Abstract

Abstract In assistive robotics, research in Brain Computer Interface aims to understand human intent to enhance Human–Robot interaction and augment human performance. In this research, a framework to enable a person with an upper limb disability to use an assistive system toward maintaining self-reliance is introduced and its implementation and evaluation are discussed. The framework interlinks functional components and establishes a behavioral sequence to operate the assistive system in three stages; action classification, verification, and execution. An action is classified based on identified human intent and verified through haptic and/or visual feedback before execution. The human intent is conveyed through facial expressions and verified through head movements. The interlinked functional components are an electro-encephalogram (EEG) sensing device, a head movement recorder, a dual-purpose glove, a visual feedback environment, and a robotic arm. Five volunteers are used to evaluate the ability of the system to recognize a facial expression, the time required to respond using head movements, convey information through vibrotactile feedback effects, and the ability to follow the established behavioral sequence. Based on the evaluation, a personalized training dataset should be used to calibrate facial expression recognition and define the time required to respond during verification. Custom vibrotactile effects were effective in conveying system information to the user. The volunteers were able to follow the behavioral sequence and control the system with a success rate of 80.00%, thus providing confidence to recruit more volunteers to identify and address improvements and expand the operational capability of the framework.