Evaluation of the Agency for Healthcare Research and Quality Pediatric Quality Indicators

Abstract

OBJECTIVES. Pediatric quality indicators were developed in 2006 by the Agency for Healthcare Research and Quality to identify potentially preventable complications in hospitalized children. Our objectives for this study were to (1) apply these algorithms to an aggregate children's hospital's discharge abstract database, (2) establish rates for each of the pediatric quality indicator events in the children's hospitals, (3) use direct chart review to investigate the accuracy of the pediatric quality indicators, (4) calculate the number of complications that were already present on admission and, therefore, not attributable to the specific hospitalization, and (5) evaluate preventability and calculate positive predictive value for each of the indicators. In addition, we wanted to use the data to set priorities for ongoing clinical investigation. METHODS. The Agency for Healthcare Research and Quality pediatric quality indicator algorithms were applied to 76 children's hospital's discharge abstract data (1794675 discharges) from 2003 to 2005. Rates were calculated for 11 of the pediatric quality indicators from all 3 years of discharge data: accidental puncture or laceration, decubitus ulcer, foreign body left in during a procedure, iatrogenic pneumothorax in neonates at risk, iatrogenic pneumothorax in nonneonates, postoperative hemorrhage or hematoma, postoperative respiratory failure, postoperative sepsis, postoperative wound dehiscence, selected infections caused by medical care, and transfusion reaction. Subsequently, clinicians from 28 children's hospitals reviewed 1703 charts in which complications had been identified. They answered questions as to correctness of secondary diagnoses that were associated with the indicator, whether a complication was already present on admission, and whether that complication was preventable, nonpreventable, or uncertain. RESULTS. Across 3 years of data the rates of pediatric quality indicators ranged from a low of 0.01/1000 discharges for transfusion reaction to a high of 35/1000 for postoperative respiratory failure, with a median value of 1.85/1000 for the 11 pediatric quality indicators. Indicators were often already present on admission and ranged from 43% for infection caused by medical care to 0% for iatrogenic pneumothorax in neonates, with a median value of 16.9%. Positive predictive value for the subset of pediatric quality indicators occurring after admission was highest for decubitus ulcer (51%) and infection caused by medical care (40%). Because of the very large numbers of cases identified and its low preventability, the indicator postoperative respiratory failure is particularly problematic. The initial definition includes all children on ventilators postoperatively for >4 days with few exclusions. Being on a ventilator for 4 days would be a normal occurrence for many children with extensive surgery; therefore, the majority of the time does not indicate a complication and makes the indicator inappropriate. CONCLUSIONS. A subset of pediatric quality indicators derived from administrative data are reasonable screening tools to help hospitals prioritize chart review and subsequent improvement projects. However, in their present form, true preventability of these complications is relatively low; therefore, the indicators are not useful for public hospital comparison. Identifying which complications are present on admission versus those that occur within the hospitalization will be essential, along with adequate risk adjustment, for any valid comparison between institutions. Infection caused by medical care and decubitus ulcers are clinically important indicators once the present-on-admission status is determined. These complications cause significant morbidity in hospitalized children, and research has shown a high level of preventability. The pediatric quality indicator software can help children's hospitals objectively review their cases and target improvement activities appropriately. The postoperative-respiratory-failure indicator does not represent a complication in the majority of cases and, therefore, should not be included for hospital screening or public comparison. Chart review should become part of the development process for quality indicators to avoid inappropriate conclusions that misdirect quality-improvement resources.