Interrater Reliability of the Simulation Fidelity (SiFi) Scale

Abstract

A limitation of the extant research on the training effectiveness of high versus low fidelity simulation is the lack of a standardized methodology to define high and low fidelity in way that is generalizable across studies. This paper presents a novel approach to standardizing the definition of fidelity using The Simulation Fidelity (SiFi) scale. The SiFi scale is a 6-point scale used to rate the fidelity of a simulation. The present study examines the inter-rater reliability of the SiFi scale to define the fidelity of simulated task elements with respect to their real-world referent. Two subject matter experts (SMEs) completed a series of aviation maneuvers in a virtual reality (VR) simulator. Following the maneuvers, each SME provided 117 ratings of simulated elements using the SiFi scale. Cohen’s k was calculated to determine interrater reliability. Results of this analysis indicate strong reliability of the SiFi scale.