Indices of Severity for Emergency Medical Evaluative Studies: Reliability, Validity, and Data Requirements

Abstract

Indices of severity, that is, numerical ratings denoting the probability of mortality or morbidity resulting from a critical illness or injury, are important research, programmatic, and clinical tools. They are essential techniques for making screening decisions in the pre-hospital and emergency department arenas, describing the impact of environmental hazards, allocating and evaluating emergency medical resources, and assessing the effectiveness of emergency care. In view of these important roles, such indices should meet minimal standards of methodologic rigor. Three sets of methodologic requirements—reliability, validity, and data requirements—are suggested and used to evaluate the methodologic adequacy of 16 commonly used severity indices. In general, the indices were found to depend on clinically subjective judgments, lacked clear or objective definitions, and offered little evidence that they could be used reliably. While all correlated with mortality, few had a documented ability to predict morbidity. Most indices had not been validated prospectively and retrospectively, nor been tested in different settings. Thus, they may be substantially contaminated by the idiosyncratic patient mix in the facility and series used for scale development, and not generalizable as a severity measure to other populations. Only five of the 16 indices met even minimal methodologic criteria. It is likely, therefore, that the methodologic basis of most existing and commonly used indices of severity is insufficient to support the heavy burden of their current usage. It is concluded that a single severity index is neither desirable nor possible, and recommendations are made for future developmental work for multiple indices each predicting separate outcomes within different time slices with varying information needs and differential predictive capacities.