Are Standard Diagnostic Test Characteristics Sufficient for the Assessment of Continual Patient Monitoring?

Abstract

Background. For diagnostic processes involving continual measurements from a single patient, conventional test characteristics, such as sensitivity and specificity, do not consider decision consistency, which might be a distinct, clinically relevant test characteristic. Objective. The authors investigated the performance of a decision-support classifier for the diagnosis of traumatic injury with blood loss, implemented with three different data-processing methods. For each method, they computed standard diagnostic test characteristics and novel metrics related to decision consistency and latency. Setting. Prehospital air ambulance transport. Patients. A total of 557 trauma patients. Design. Continually monitored vital-sign data from 279 patients (50%) were randomly selected for classifier development, and the remaining were used for testing. Three data-processing methods were evaluated over 16 min of patient monitoring: a 2-min moving window, time averaging, and postprocessing with the sequential probability ratio test (SPRT). Measurements. Sensitivity and specificity were computed. Consistency was quantified through cumulative counts of decision changes over time and the fraction of patients affected by false alarms. Latency was evaluated by the fraction of patients without a decision. Results. All 3 methods showed very similar final sensitivities and specificities. Yet, there were significant differences in terms of the fraction of patients affected by false alarms, decision changes through time, and latency. For instance, use of the SPRT led to a 75% reduction in the number of decision changes and a 36% reduction in the number of patients affected by false alarms, at the expense of 3% unresolved final decisions. Conclusion. The proposed metrics of decision consistency and decision latency provided additional information beyond what could be obtained from test sensitivity and specificity and are likely to be clinically relevant in some applications involving temporal decision making.