Inter-Rater Reliability of Delirium Screening of Infants in the Cardiac ICU: A Prospective, Observational Study

Abstract

OBJECTIVES: To determine the inter-rater reliability (IRR) of the Cornell Assessment for Pediatric Delirium (CAP-D) in infants admitted to a cardiac ICU (CVICU) and to explore the impact of younger age and mechanical ventilation on IRR. DESIGN: Prospective cross-sectional study of delirium screening performed by bedside CVICU nurses. We collected data from September 2020 to April 2021. We evaluated IRR with intraclass correlation coefficient (ICC) one-way random effects and Fleiss kappa for multiple raters. SETTING: Eighteen-bed academic pediatric CVICU. PARTICIPANTS: Subjects: Infants 1 day to 1 year old admitted to the CVICU, stratified in two age groups (≤ 9 wk and 9 wk to < 1 yr). Exclusion criteria were patients’ immediate postoperative day, State Behavioral Scale score less than or equal to –2, or at risk for hemodynamic instability with assessment. Raters: CVICU nurses working in the unit during study days. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Groups of four raters performed 91 assessments, a total of 364 CAP-D screens. Forty-five of 91 (49%) were in patients less than or equal to 9 weeks old and 43 of 91 (47%) in mechanically ventilated patients. Sixty-eight of 81 nurses (81%) participated. In infants less than or equal to 9 weeks old, ICC was 0.59 (95% CI 0.44–0.71), poor to moderate reliability, significantly lower than the ICC in infants greater than 9 weeks and 0.72 (95% CI 0.61–0.82), moderate to good reliability. In mechanically ventilated infants, ICC was 0.5 (95% CI 0.34–0.65), poor to moderate reliability, significantly lower than the ICC in nonmechanically ventilated infants and 0.69 (95% CI 0.57–0.8), moderate to good reliability. Fleiss kappa for all infants was 0.47 (95% CI 0.34–0.6), slight to fair agreement. Use of anchor points did not improve reliability. CONCLUSIONS: In the youngest, most vulnerable infants admitted to the CVICU, further evaluation of the CAP-D tool is needed.