Prospective Evaluation of a Dynamic Acuity Score for Regularly Assessing a Critically Ill Patient’s Risk of Mortality

Abstract

OBJECTIVE: Predictive models developed for use in ICUs have been based on retrospectively collected data, which does not take into account the challenges associated with live, clinical data. This study sought to determine if a previously constructed predictive model of ICU mortality (ViSIG) is robust when using data collected prospectively in near real-time. DESIGN: Prospectively collected data were aggregated and transformed to evaluate a previously developed rolling predictor of ICU mortality. SETTING: Five adult ICUs at Robert Wood Johnson-Barnabas University Hospital and one adult ICU at Stamford Hospital. PATIENTS: One thousand eight hundred and ten admissions from August to December 2020. MEASUREMENTS AND MAIN RESULTS: The ViSIG Score, comprised of severity weights for heart rate, respiratory rate, oxygen saturation, mean arterial pressure, mechanical ventilation, and values for OBS Medical’s Visensia Index. This information was collected prospectively, whereas data on discharge disposition was collected retrospectively to measure the ViSIG Score’s accuracy. The distribution of patients’ maximum ViSIG Score was compared with ICU mortality rate, and cut points determined where changes in mortality probability were greatest. The ViSIG Score was validated on new admissions. The ViSIG Score was able to stratify patients into three groups: 0–37 (low risk), 38–58 (moderate risk), and 59–100 (high risk), with mortality of 1.7%, 12.0%, and 39.8%, respectively (p < 0.001). The sensitivity and specificity of the model to predict mortality for the high-risk group were 51% and 91%. Performance on the validation dataset remained high. There were similar increases across risk groups for length of stay, estimated costs, and readmission. CONCLUSIONS: Using prospectively collected data, the ViSIG Score produced risk groups for mortality with good sensitivity and excellent specificity. A future study will evaluate making the ViSIG Score visible to clinicians to determine whether this metric can influence clinician behavior to reduce adverse outcomes.