Validity of Agency for Healthcare Research and Quality Patient Safety Indicators at an Academic Medical Center

Abstract

The Agency for Healthcare Research and Quality developed Patient Safety Indicators (PSI) to screen for in-hospital complications and patient safety events through International Classification of Diseases, 9th Revision, Clinical Modification coding. The purpose of this study was to validate 10 common surgically related PSIs at our academic medical center and investigate the causes for inaccuracies. We reviewed patient records between October 2011 and September 2012 at our urban academic medical center for 10 common surgically related PSIs. The records were reviewed for incorrectly identified PSIs and a subset was further reviewed for the contributing factors. There were 93,169 charts analyzed for PSIs and 358 PSIs were identified (3.84 per 1000 cases). The overall positive predictive value (PPV) was 83 per cent (95% confidence interval 79 to -86%). The lowest PPVs were associated with catheter-related bloodstream infections (67%), postoperative respiratory failure (71%), and pressure ulcers (79%). The most common contributing factors for incorrect PSIs were coding errors (30%), documentation errors (19%), and insufficient criteria for PSI in the chart (16%). We conclude that the validity of PSIs is low and could be improved by increased education for clinicians and coders. In their current form, PSIs remain suboptimal for widespread use in public reporting and pay-for-performance evaluation.