Does Situation Awareness Add to the Validity of Cognitive Tests?

Abstract

Objective: Does adding situation awareness (SA) to a battery of cognitive tests improve prediction? Background: Identifying variables that predict skilled performance in a complex task aids in understanding the nature of skill and also aids in the selection of operators to perform that task. SA is thought to be an important predictor of performance. SA is often thought to be based on underlying cognitive mechanisms. Method: Three performance measures taken from the Federal Aviation Administration's (FAA's) Air Traffic Scenarios Test, the low-fidelity simulation component of the FAA's controller selection battery, were used as criterion variables in a hierarchical regression. After predicting performance based on a battery of cognitive (e.g., intelligence, working memory, spatial memory) and noncognitive tests (e.g., cognitive style, personality, demographics), we added measures of SA. Results: SA did provide increases in prediction, but only when measured with the Situation Present Assessment Method, an on-line query method. When the same questions were asked off line, SA did not enter the model in two cases and improved prediction by only 2% in the third. Conclusion: Thus, some measures of SA do show incremental validity, even against a backdrop of a large number of cognitive variables. Application: On-line measures of SA can be a worthwhile addition to standard batteries of tests used to predict performance in cognitively oriented industrial tasks.