Interrater Reliability of APACHE II Scores for Medical-Surgical Intensive Care Patients: A Prospective Blinded Study

Abstract

Background Despite widespread use of the Acute Physiology and Chronic Health Evaluation II (APACHE II), its interrater reliability has not been well studied. Objective To determine interrater reliability of APACHE II scores among 1 intensive care nurse and 2 research clerks. Methods In a prospective, blinded, observational study, 3 raters collected APACHE II scores on 37 consecutive patients in a medical-surgical intensive care unit. One research clerk was blinded to the study’s start date to minimize observer bias. The nurse and the other research clerk were blinded to each other’s scores and did not communicate with the first research clerk about the study. The data analyst was blinded to the identity and source of all 3 raters’ scores. Intraclass correlation coefficients and 95% confidence intervals were assessed. Results Mean (standard deviation) APACHE II scores were 21.8 (9.2) for the nurse, 20.4 (7.7) for research clerk 1, and 20.5 (8.1) for research clerk 2. Among the 3 raters, the intraclass correlation coefficient (95% confidence interval) was 0.90 (0.84, 0.94) for the APACHE II total score. Within APACHE II score components, the highest reliability was for age (0.98 [0.97, 0.99]), with lower reliabilities for the Chronic Health Index (0.64 [0.50, 0.80]) and the verbal component of the Glasgow Coma Scale (0.40 [0.20, 0.60]). Results were similar between pairs of raters. Conclusions Use of trained nonmedical personnel to collect illness severity scores for clinical, research, and administrative purposes is reasonable. This method could be used to assess reliability of other illness severity scores.