To What Extent Should Quality of Care Decisions Be Based on Health Outcomes Data?

Abstract

Background and Purpose— Most quality improvement methods implicitly assume that facilities with high complication rates are likely to have substandard processes of care, a stable characteristic that, in the absence of intervention, will persist over time. We assessed the extent to which this holds true for carotid endarterectomy. Methods— Using data from the Department of Veterans Affairs National Surgical Quality Improvement Project, we classified facilities on the basis of 30-day complications of carotid endarterectomy (stroke, myocardial infarction, death) during 1994 to 1995 (period 1, n=3389) and then compared these groups of facilities for complication rates during 1996 to 1997 (period 2, n=4453). Results— Despite wide variation in facility-specific complication rates, the correlation between rates in periods 1 and 2 was low (Spearman correlation coefficient, 0.04; P =0.01) Facility-specific rates did not show greater correlation when we examined only facilities with higher volumes patients in different clinical categories (asymptomatic, transient ischemic attack, stroke). Comorbid illness profiles were similar between the 2 time periods. Conclusions— Most of the facility-specific differences in complication rates in period 1 were not maintained into period 2. Many apparent quality improvement problems may not be as large as they first appear, especially when based on few complications per facility. The inability, in practice, to estimate complication rates at a high degree of precision is a fundamental difficulty for clinical policy making regarding procedures with complication rates such as carotid endarterectomy.