Development and Reliability Review of an Assessment Tool to Measure Competency in the Seven Elements of the Risk Management Process: Part Two—Evaluation of the Individual Results from the RISKometric Tool

Abstract

This study aims to test further the RISKometric previously developed by the authors. This paper is the second of three studies: it compares individuals’ RISKometric results in the first study with their performance in a risk scenario exercise in this second study; so, providing a reliability review for the RISKometric. A risk scenario exercise was developed that required participants to individually undertake a risk management process on a realistic, potentially hazardous event involving working at heights during simultaneous operations. Two observers assessed their responses, rating the participants’ competence in each of the seven risk management process elements. Twenty-six participants individually undertook the risk scenario exercise, known as round one. Analyses found that participants’ individual competence ratings given to them (by observers) when undertaking the risk scenario exercise were strongly and positively associated with the competence ratings given to them by their peers and downline colleagues in the RISKometric in an earlier study; for each of the seven elements in this second study. This finding supports the RISKometric as a useful tool for rating the competency of individuals in the seven elements of the risk management process. Work was also undertaken in preparation for a planned future third study whereby eight participants of the original 26 were selected to individually undertake the risk scenario exercise again to determine any difference in ratings, e.g., if there was a learning effect. The analysis found no significant difference over the two rounds.