A systematic review and evaluation of Zika virus forecasting and prediction research during a public health emergency of international concern

Abstract

AbstractINTRODUCTIONEpidemic forecasting and prediction tools have the potential to provide actionable information in the midst of emerging epidemics. While numerous predictive studies were published during the 2016-2017 Zika Virus (ZIKV) pandemic, it remains unknown how timely, reproducible and actionable the information produced by these studies was.METHODSTo improve the functional use of mathematical modeling in support of future infectious disease outbreaks, we conducted a systematic review of all ZIKV prediction studies published during the recent ZIKV pandemic using the PRISMA guidelines. Using MEDLINE, EMBASE and grey literature review, we identified studies that forecasted, predicted or simulated ecological or epidemiological phenomenon related to the Zika pandemic that were published as of March 01, 2017. Eligible studies underwent evaluation of objectives, data sources, methods, timeliness, reproducibility, accessibility and clarity by independent reviewers.RESULTS2034 studies were identified, of which n = 73 met eligibility criteria. Spatial spread, R0 (basic reproductive number) and epidemic dynamics were most commonly predicted, with few studies predicting Guillain-Barré Syndrome burden (4%), sexual transmission risk (4%) and intervention impact (4%). Most studies specifically examined populations in the Americas (52%), with few African-specific studies (4%). Case count (67%), vector (41%) and demographic data (37%) were the most common data sources. Real-time internet data and pathogen genomic information were used in 7% and 0% of studies, respectively, and social science and behavioral data were typically absent in modeling efforts. Deterministic models were favored over stochastic approaches. Forty percent of studies made model data entirely available, 29% provided all relevant model code, 43% presented uncertainty in all predictions and 54% provided sufficient methodological detail allowing complete reproducibility. Fifty-one percent of predictions were published after the epidemic peak in the Americas. While the use of preprints improved the accessibility of ZIKV predictions by a median 119 days sooner than journal publication dates, they were used in only 30% of studies.CONCLUSIONSMany ZIKV predictions were published during the 2016-2017 pandemic. The accessibility, reproducibility, timeliness, and incorporation of uncertainty in these published predictions varied and indicates that there is substantial room for improvement. To enhance the utility of analytical tools for outbreak response, it is essential to improve the sharing of model data, code, and preprints for future outbreaks, epidemics and pandemics.Author summaryResearchers published many studies which sought to predict and forecast important features of Zika virus (ZIKV) infections and their spread during the 2016-2017 ZIKV pandemic. We conducted a comprehensive review of such ZIKV prediction studies and evaluated their aims, the data sources they used, which methods were used, how timely they were published, and whether they provided sufficient information to be used or reproduced by others. Of the 73 studies evaluated, we found that the accessibility, reproducibility, timeliness, and incorporation of uncertainty in these published predictions varied and indicates that there is substantial room for improvement. We identified that the release of study findings before formal journal publication (‘pre-prints’) increased the timeliness of Zika prediction studies, but note they were infrequently used during this public health emergency. Addressing these areas can improve our understanding of Zika and other outbreaks and ensure that forecasts can inform preparedness and response to future outbreaks, epidemics and pandemics.