Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error

Abstract

Abstract We investigate the effect of estimation error on backtests of expected shortfall (ES) forecasts. These backtests are based on first-order conditions of a recently introduced family of jointly consistent loss functions for value-at-risk (VaR) and ES. For both single and multiperiod horizons, we provide explicit expressions for the additional terms in the asymptotic covariance matrix that result from estimation error, and propose robust tests that account for it. Monte Carlo experiments show that the tests that ignore these terms suffer from size distortions, which are more pronounced for higher ratios of out-of-sample to in-sample observations. Robust versions of the backtests perform well with power against common alternatives. We also introduce a novel standardization of the conditional joint test statistic that removes the need to estimate higher-order moments and significantly improves its performance. In an application to VaR and ES forecasts for daily FTSE 100 index returns as generated by (GJR-)GARCH and HEAVY models, we find that estimation error substantially impacts the outcome of the backtests, and is not bound to particular subperiods such as the credit crisis.