SDE-HNN: Accurate and Well-calibrated Forecasting using Stochastic Differential Equations

Abstract

It is crucial yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. Heteroscedastic neural networks (HNNs) are promising methods that capture data uncertainty for forecasting problems, while existing HNNs have difficulties in conjoining calibrated uncertainty estimation and satisfactory predictive performance due to the failure to construct an explicit interaction between the prediction and its associated uncertainty. This paper develops SDE-HNN, an improved HNN equipped with stochastic differential equations (SDE), to characterize the interaction between the predictive mean and variance inside HNNs for accurate and reliable forecasting. The existence and uniqueness of the solution to the devised neural SDE are guaranteed. Moreover, based on the bias-variance trade-off for the optimization in SDE-HNN, we design an enhanced numerical SDE solver to improve learning stability. Finally, we present two new diagnostic uncertainty metrics to systematically evaluate the predictive uncertainty. Experiments on various challenging datasets show that our method significantly outperforms state-of-the-art baselines on both predictive performance and uncertainty quantification, delivering well-calibrated and sharp prediction intervals in time-series forecasting.