Interpretable Models for Near-real-time Prediction of Team Cognitive Workload in Complex Sociotechnical Environments Using Behavioral and Physiological Data

Abstract

This work develops interpretable models to predict near-real-time cognitive workload (CWL) in teams operating in complex environments. Existing approaches using neurological sensors are impractical for field use. Our approach integrates multimodal data from non-invasive behavioral and physiological sensors to robustly detect CWL changes. We apply multidimensional recurrence quantification analysis (MdRQA) with a novel pattern analysis extension to identify recurring multimodal signatures indicative of different CWL states. A multiparty dataset with fNIRS, behavioral, and physiological measures from teams performing a gamified search and rescue mission and individual resting state tasks were used. The findings indicate that the multimodal patterns derived from non-invasive measures were significantly associated with a neurological measure of CWL within 10s time slices. Moreover, the multimodal patterns were predictive of individual and team performance over-and-above the neurological measure of CWL. This can enable timely interventions by intelligent systems to optimally manage team CWL and enhance human-machine teaming in demanding environments.