Differential behaviour of a risk score for emergency hospital admission by demographics in Scotland — a retrospective study

Abstract

AbstractThe Scottish Patients at Risk of Re-Admission and Admission (SPARRA) score predicts individual risk of emergency hospital admission for approximately 80% of the Scottish population. It was developed using routinely collected electronic health records, and is used by primary care practitioners to inform anticipatory care, particularly for individuals with high healthcare needs. We comprehensively assess the SPARRA score across population subgroups defined by age, sex, ethnicity, socioeconomic deprivation, and geographic location. For these subgroups, we consider differences in overall performance, score distribution, and false positive and negative rates, using causal methods to identify effects mediated through age, sex, and deprivation. We show that the score is well-calibrated across subgroups, but that rates of false positives and negatives vary widely, mediated by a range of causes. Our work assists practitioners in the application and interpretation of the SPARRA score in population subgroups.Research in contextEvidence before this studyThere is considerable literature on the general topic of differential performance of risk scores across population subgroups and its implications. A shared theme is the importance of identifying and quantifying such differential performance. We performed a MedLine and Google Scholar search with the single term ‘SPARRA’, and consulted colleagues at Public Health Scotland about any previous internal analyses. Several articles assessed the accuracy of SPARRA and discussed its role in the Scottish healthcare system since its introduction in 2006, but none looked in detail at differential performance between specific demographic groups.Added value of this studyWe provide a comprehensive assessment of the performance of the SPARRA score across a range of population subgroups in several ways. We systematically examined differences in performance using a range of metrics. We identify notable areas of differential performance associated with age, sex, socioeconomic deprivation, ethnicity and residence location (mainland versus island; urban versus rural). We also examined the pattern of errors in prediction across medical causes of emergency admission, finding that, to variable degrees across groups, cardiac and respiratory admissions are more likely to be correctly predicted from electronic health records. Overall, our work provides an atlas of performance measures for SPARRA and partly explains how between-group performance differences arise.Implications of all the available evidenceThe precision by which the SPARRA score can predict emergency hospital admissions differs between population subgroups. These differences are largely driven by variation in performance across age and sex, as well as the predictability of different causes of admission. Awareness of these differences is important when making decisions based on the SPARRA score.