A Three-Step Reliability Strategy Applied to Police-Worn Body Camera Footage

Abstract

Strong inferences are drawn from police-worn body camera (BWC) footage, frequently without an assessment of reliability. Unique characteristics of BWC footage (i.e., capturing trends and less frequent behavior, a focal actor (officer) is absent) suggest a specific reliability strategy. A three-step strategy of selecting appropriate reliability indexes, providing salient reliability categories, and ranking the reliability categories was applied to BWC footage. Five interrater agreement ( $\text{A}{\text{C}}_1$ , ${\widehat{\alpha }}_K$ , $B-P$ coefficient, $r_{\text{wg}}$ , $\text{ad}.\text{m}$ ), two interrater consistency $\left(\text{ICC}\left(1,1\right),\text{ICC}\left(2,1\right)\right)$ , and three internal consistency ( ${\omega }_t$ , ${\omega }_h$ , $\text{GLB}$ ) indexes were applied to police BWC footage. A focus was to ascertain the upper limits of reliability for BWC footage. Item development and rater training were conducted to optimize rating reliability. Using a within design and confidence intervals, the relatively stronger and weaker reliabilities across the six domains of video completeness, respect (passive, active, discourse), threats, and behavioral stance were assessed. Applied to the admissibility of court evidence, central aspects of video completeness have relatively stronger reliabilities. For research, lower reliabilities have a cost of limited generalizability and ecological validity. Policy recommendations include the usage of a standardized scale with multiple ratings to determine what information should be used in high-stake decision-making based on BWC footage. The three-step strategy integrated the reliability indexes into a single figure to reflect a reliability summary of each component of BWC footage. Weighted rankings found the Overall Audio Quality (-4.9) and Empathy (-4.9) items to have the weakest reliabilities and the Clarification (5.1) and Physical Resistance (4.9) items to have the strongest reliabilities.