Evaluating Operator Training Performance Using Recurrence Quantification Analysis of Autocorrelation Transformed Eye Gaze Data

Abstract

Objective This research aimed to investigate the relationship between gaze behaviour dynamics and operator performance. Background Individuals differ in their approach when learning a new task often resulting in performance disparity. During training some individuals learn the structure and dynamics of the task and develop a systematic approach, whereas others may achieve the same result albeit with increased perceived workload, or indeed some may fail to achieve superior performance levels. Previous research has shown that comparing gaze of experts with novices can provide unique insights into cognitive functioning of superior performers. Methods Twenty-five individuals participated in a computer-based simulation task. The concept of coefficient of variation (CoV) of task scores was used to compute the participants’ consistency of performance. Based on CoV, the cohort was split into two performance categories. The temporal patterns in participants gaze data were transformed using autocorrelation, and recurrence quantification analysis (RQA) was employed to analyse and quantify the patterns. Results A Mann–Whitney U analysis demonstrated significantly ( p < .01) higher determinism, entropy and laminarity in the superior group compared to the moderate group. Pearson’s correlation revealed a significant ( p < .01) negative correlation between the consistency of task performance (CoV) and the RQA measures. Conclusion The results demonstrated that eye gaze dynamics can be used as an objective measure of performance. Participants classified as superior performers consistently demonstrated a systematic gaze activity which were in line with the task structure. Application The methods presented here are applicable to observe and evaluate operators’ strategic distribution of gaze. Specifically, for tactical monitoring and decision making in task environments where spatial locations of elements-of-interest vary continuously.