Inter- and Intra-Rater Reliabilities of the Army Combat Fitness Test Three-Repetition Maximum Deadlift Event Among Raters of Varying Professional Experience

Abstract

ABSTRACT Introduction The U.S. Army is updating the physical fitness assessment for soldiers to the six-event Army Combat Fitness Test (ACFT). A paucity of data regarding the ACFT maximum deadlift (MDL) event, especially in military populations, has increased concern over the objectivity of the test. The reliability of scoring the MDL has not been established. It is unknown if grader professional experience impacts the reliability of scoring, and if so, what level of experience is required for reliable assessment. Performance and assessment of the MDL could impact military occupational selection, promotion, and retention within the Army. The purposes of this study were to determine the inter- and intra-rater reliabilities of raters with varying degrees of professional experience on scoring the MDL and to determine the relationships between load lifted, overall lift success, sex, and body mass index (BMI). Methods The design is a reliability study. Approval was granted by the Naval Medical Center—Portsmouth Institutional Review Board. Fifty-five healthy soldiers and cadets from the U.S. Military Academy were recruited. Participants completed one data collection session, performing one MDL attempt. The attempt was video recorded using three devices: two handheld tablets placed perpendicular to the sagittal and frontal planes recording at 240 Hz and one digital camera positioned at a 45° angle recording at 30 Hz. A reference standard was established through slow-motion analysis of the sagittal and frontal plane recordings. Six raters with varying degrees of professional experience viewed the 45° camera recordings at real-time speed independently, in a random order, on two separate occasions. Lift success was dichotomously assessed as successful or unsuccessful according to the MDL standards. Cohen’s kappa was computed to determine inter- and intra-rater reliabilities among raters. Bivariate correlation was used to assess associations among load lifted, BMI, and sex. A chi-squared test of independence assessed the relationship between sex and overall lift success. Results Inter-rater reliability between the six raters ranged from 0.29 to 0.69. Inter-rater reliability between the raters to the reference standard ranged from 0.28 to 0.61. Intra-rater reliability ranged from 0.51 to 0.84. Inter-rater reliability of raters who had attended a Training and Doctrine Command–approved ACFT certification course ranged from 0.51 to 0.66, while those who had not ranged from 0.34 to 0.46. BMI and sex were associated with load lifted (r = 0.405, P = .002; r = −0.727, P < .001, respectively). Overall lift success was not associated with load lifted (r = −0.047, P = .731). Overall lift success was not related to sex (χ2 = 0.271, P = .602). Conclusion Inter-rater reliability of the six raters ranged from poor to substantial, while intra-rater reliability ranged from moderate to excellent. Compared to a reference standard, inter-rater reliability ranged from poor to substantial. The wide range in consistency demonstrated in this study, both between and within raters, brings into question the current subjective methods used to grade the MDL. More research is needed to understand the most feasible, valid, and reliable way to assess performance standards like the MDL that may affect a soldier’s career progression.