Combining Process Tracing and Policy Capturing Techniques for Judgment Analysis in an Anti-Submarine Warfare Simulation

Abstract

The Cognitive Shadow is a prototype decision support tool that can notify users when they deviate from their usual judgment pattern. Expert decision policies are learned automatically online while performing one’s task using a combination of machine learning algorithms. This study investigated whether combining this system with the use of a process tracing technique could improve its ability to model human decision policies. Participants played the role of anti-submarine warfare commanders and rated the likelihood of detecting a submarine in different ocean areas based on their environmental characteristics. In the process tracing condition, participants were asked to reveal only the information deemed necessary, and only that information was sent to the system for model training. In the control condition, all the available information was sent to the system with each decision. Results showed that process tracing data improved the model’s ability to predict human decisions compared to the control condition.