Interrater reliability of chart-based assessment of functional impairment after pediatric injury using the functional status scale

Abstract

BACKGROUND Functional impairment has been proposed as an alternative outcome for quality improvement in pediatric trauma. The functional status scale (FSS) has been used in studies of injured children, but has only been validated with resource-intensive in-person assessment. Implementation with retrospective chart-based FSS assessment would offer a simplified and scalable alternative. The purpose of this study was to evaluate interrater reliability of retrospective FSS assessment and to identify factors associated with unreliable assessment. METHODS A retrospective cohort of admissions to a Level I pediatric trauma center between July 2020 and June 2021 was analyzed. Two physicians and two nurse registrars reviewed charts to obtain measures of six FSS domains (mental status, sensory functioning, communication, motor functioning, feeding, and respiratory status) at discharge. Functional impairment was categorized by total FSS scores as good (6,7), mild impairment (8,9), moderate impairment (10–15), severe impairment (16–21), or very severe impairment (>21). Interrater reliability was assessed using intraclass correlation (ICC). Predictors of rater disagreement were evaluated using multivariable logistic regression. RESULTS The cohort included 443 children with a mean age of 7.4 years (standard deviation, 5.4 years) and median Injury Severity Score of 9 (interquartile range, 5–12). The median time per chart to assess FSS was 2 minutes (interquartile range, 1–2). Thirty-seven patients (8%) had functional impairment at discharge. Interrater reliability was excellent for total FSS score (ICC = 0.87) and good for FSS impairment categorization (ICC = 0.80). Rater disagreement of functional impairment categorization occurred in 14% of cases overall. Higher level of functional impairment and use of therapies (occupational and speech language therapy) were independently associated with more frequent rater disagreement. CONCLUSION Chart-based FSS assessment is feasible and reliable, but may require more detailed review for patients with higher level of impairment that require allied health therapy. Validation of chart-based assessment is needed before widespread implementation. LEVEL OF EVIDENCE Prognostic/Epidemiological, Level III.