Recalibrating probabilistic forecasts of epidemics
-
Published:2022-12-15
Issue:12
Volume:18
Page:e1010771
-
ISSN:1553-7358
-
Container-title:PLOS Computational Biology
-
language:en
-
Short-container-title:PLoS Comput Biol
Author:
Rumack AaronORCID,
Tibshirani Ryan J.ORCID,
Rosenfeld RoniORCID
Abstract
Distributional forecasts are important for a wide variety of applications, including forecasting epidemics. Often, forecasts are miscalibrated, or unreliable in assigning uncertainty to future events. We present a recalibration method that can be applied to a black-box forecaster given retrospective forecasts and observations, as well as an extension to make this method more effective in recalibrating epidemic forecasts. This method is guaranteed to improve calibration and log score performance when trained and measured in-sample. We also prove that the increase in expected log score of a recalibrated forecaster is equal to the entropy of the PIT distribution. We apply this recalibration method to the 27 influenza forecasters in the FluSight Network and show that recalibration reliably improves forecast accuracy and calibration. This method, available on Github, is effective, robust, and easy to use as a post-processing tool to improve epidemic forecasts.
Funder
Center for Machine Learning and Health at Carnegie Mellon
McCune Charitable Foundation
Centers for Disease Control and Prevention
Publisher
Public Library of Science (PLoS)
Subject
Computational Theory and Mathematics,Cellular and Molecular Neuroscience,Genetics,Molecular Biology,Ecology,Modeling and Simulation,Ecology, Evolution, Behavior and Systematics
Reference25 articles.
1. Reliability, sufficiency, and the decomposition of proper scores;J Bröcker;Quarterly Journal of the Royal Meteorological Society,2009
2. Calibration-based empirical probability;AP Dawid;Annals of Statistics,1985
3. Probabilistic forecasts, calibration and sharpness;T Gneiting;Journal of the Royal Statistical Society, Series B,2007
4. Strictly proper scoring rules, prediction, and estimation;T Gneiting;Journal of the American Statistical Association,2007
5. Probability judgements for continuous quantities;SC Hora;Management Science,2004
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献