Regularized COVID-19 Forecast Ensemble Methods

Abstract

AbstractForecasts of COVID-19 outcomes play an essential role in alerting public health and government officials to the trajectory of the pandemic. The sudden and critical need for these forecasts spurred both the proliferation of diverse epidemiological transmission models from academia and industry across the United States and efforts to standardize and curate these model outputs. In many scientific domains, ensemble models, where individual forecasts are aggregated into one, have demonstrated smaller forecasting error than the individual models from which they are constructed. Using COVID-19 deaths as an index outcome, we developed and evaluated several ensemble approaches where point forecast models were combined via weighted sums based on historical individual model or ensemble model performance. We found that a simple method that minimized the error of the past performance of individual models and used L2 regularization to encourage broader distribution of weights across models outperformed a baseline mean ensemble and all other tested methods across US states for both absolute error and weighted interval scores. This suggests that performance-based ensembles can produce accurate forecasts despite training on only point forecasts and recent historical data, provided that sufficient regularization and constraints are used to capture uncertainty. Availability of an accurate and explainable ensemble forecast model can increase trust among stakeholders and the general public, thus bettering preparedness and response efforts during the COVID-19 pandemic.