Agent Reasoning Transparency’s Effect on Operator Workload

Abstract

We examined how varying the transparency of agent reasoning affected operator workload in a route selection task, and how the differing measures of workload compared in assessing and understanding cognitive workload. Participants guided a three-vehicle convoy safely through a simulated environment of which they had a limited amount of information, while maintaining communication with command and monitoring their surroundings for threats. The intelligent route-planning agent assessed potential threats and suggested changes to the convoy route as needed. Each participant was assigned to one of three agent reasoning transparency conditions. Contrary to our hypothesis, NASA-TLX Global workload measures indicated that workload decreased slightly as access to agent reasoning increased. However, psychophysical measures of workload disagreed with NASA-TLX global results. Comparison of individual NASA-TLX workload factors with the psychophysical measures indicated that performance satisfaction was highest in the intermediary transparency condition, and the addition of ambiguous information in the highest transparency condition increased effort and resulted in increased complacent behavior. Recommendations for future workload analysis are offered.