Dynamic Cognitive Load Assessment in Virtual Reality

Abstract

Background Recent advances in non-invasive physiologic monitoring leverage machine learning to provide unobtrusive, real-time assessments of a learner’s cognitive load (CL) as they engage in specific tasks. However, the performance characteristics of these novel composite physiologic CL measures are incompletely understood. Objectives We aimed to 1) explore the feasibility of measuring CL in real time using physiologically-derived inputs; 2) evaluate the performance characteristics of a novel composite CL measure during simulated virtual reality resuscitations; and 3) understand how this measure compares to traditional, self-reported measures of CL. Methods Novice (PGY1-2 pediatric residents) and expert (pediatric emergency medicine fellows and attendings) participants completed four virtual reality simulations as team leader. The scenario content (status epilepticus versus anaphylaxis) and level of distraction (high versus low) were manipulated. Cognitive load was measured in all participants using electroencephalography and electrocardiography data (“real-time CL”) as well as through self-report (NASA-TLX). Scenario performance also was measured. Results Complete data were available for 6 experts and 6 novices. Experts generally had lower CL than novices on both measures. Both measures localized the most significant differences between groups to the anaphylaxis scenarios (real-time CL [low-distraction] Cohen’s d -1.33 [95% CI -.2.56, -0.03] and self-reported CL [high-distraction] Cohen’s d -1.41 [95% CI -2.67, -0.10]). No consistent differences were seen with respect to level of distraction. Performance was similar between the two groups, though both exhibited fewer errors over time (F(3,48) = 5.75, p = .002). Conclusion It is feasible to unobtrusively measure cognitive load in real time during virtual reality simulations. There was convergence between the two CL measures: in both, experts had lower CL than novices, with the most significant effect size differences in the more challenging anaphylaxis scenarios.