Intra- and Inter-Rater Reliability of Manual Feature Extraction Methods in Movement Related Cortical Potential Analysis

Abstract

Event related potentials (ERPs) provide insight into the neural activity generated in response to motor, sensory and cognitive processes. Despite the increasing use of ERP data in clinical research little is known about the reliability of human manual ERP labelling methods. Intra-rater and inter-rater reliability were evaluated in five electroencephalography (EEG) experts who labelled the peak negativity of averaged movement related cortical potentials (MRCPs) derived from thirty datasets. Each dataset contained 50 MRCP epochs from healthy people performing cued voluntary or imagined movement, or people with stroke performing cued voluntary movement. Reliability was assessed using the intraclass correlation coefficient and standard error of measurement. Excellent intra- and inter-rater reliability was demonstrated in the voluntary movement conditions in healthy people and people with stroke. In comparison reliability in the imagined condition was low to moderate. Post-hoc secondary epoch analysis revealed that the morphology of the signal contributed to the consistency of epoch inclusion; potentially explaining the differences in reliability seen across conditions. Findings from this study may inform future research focused on developing automated labelling methods for ERP feature extraction and call to the wider community of researchers interested in utilizing ERPs as a measure of neurophysiological change or in the delivery of EEG-driven interventions.