Can a “goldilocks” Mortality Predictor Perform Consistently Across Time and Equitably Across Populations?

Abstract

Abstract Objective: Advance care planning (ACP) facilitates end-of-life care, yet many die without one. Timely and accurate mortality prediction may encourage ACP. Therefore, we assessed performance equity and consistency for a novel 5-to-90-day mortality predictor. Methods: Predictions were made for the first day of included adult inpatient admissions on a retrospective dataset. Performance was assessed across various demographies, geographies, and timeframes. Results: AUC-PR remained at 29% both pre- and during COVID. Pre-COVID-19 recall and precision were 58% and 25% respectively at the 12.5% cutoff, and 12% and 44% at the 37.5% cutoff. During COVID-19, recall and precision were 59% and 26% at the 12.5% cutoff, and 11% and 43% at the 37.5% cutoff. Pre-COVID, recall dropped at both cutoffs if recent data was not made available to the model; and compared to the overall population, recall was lower at the 12.5% cutoff in the White, non-Hispanic subgroup and at both cutoffs in the rural subgroup. During COVID-19, precision at the 12.5% cutoff was lower than that of the overall population for the non-White and non-White female subgroups. No other statistically significant differences were seen between subgroups and the corresponding overall population. Conclusions: Overall predictive performance during the pandemic was unchanged from pre-pandemic performance. Although some comparisons (especially precision at the 37.5% cutoff) were underpowered, precision at the 12.5% cutoff was equitable across most demographies, regardless of the pandemic. Mortality prediction to prioritize ACP conversations can be provided consistently and equitably across many studied timeframes, geographies, and demographies.