An Empirical Study of Calibration in Air Traffic Control Expert Judgment

Abstract

In contrast to many studies revealing biases in the probabilistic judgments of task-naïve participants, a growing body of literature has revealed that over time, professionals are able to gain a reasonably accurate appreciation for the inherent uncertainty that exists in their work environments. The present study assessed how well experienced (working) air traffic controllers are able to predict the probability of the loss of separation between a pair of converging aircraft. Sixteen controllers expressed probabilistically whether or not the depicted pair of aircraft would lose separation. The actual probability of conflict was manipulated by varying the time differential between when each pair of aircraft would reach the point of potential conflict, coupled with uncertainty due to wind perturbations. Results revealed that in instances where perceptual information was available to distinguish between conflicts and non-conflicts, the difference between the actual conflict probability and the mean of the controllers' judged probabilities of conflict was minimal, highlighting the high calibration level of these domain experts at an aggregate level.