Validation of corporate probability of default models considering alternative use cases and the quantification of model risk

Abstract

<abstract> <p>In this study we consider the construction of <italic>through-the-cycle</italic> ("TTC") <italic>probability-of-default</italic> ("PD") models designed for credit underwriting uses and <italic>point-in-time</italic> ("PIT") PD models suitable for early warning uses, considering which validation elements should be emphasized in each case. We build PD models using a long history of large corporate firms sourced from Moody's, with a large number of financial, equity market and macroeconomic candidate explanatory variables. We construct a Merton model-style <italic>distance-to-default</italic> ("DTD") measure and build hybrid structural and reduced-form models to compare with the financial ratio and macroeconomic variable-only models. In the hybrid models, the financial and macroeconomic explanatory variables still enter significantly and improve the predictive accuracy of the TTC models, which generally lag behind the PIT models in that performance measure. While all classes of models have high discriminatory power by most measures, on an out-of-sample basis the TTC models perform better than the PIT models. We measure the model risk attributable to various model assumptions according to the principle of <italic>relative entropy</italic> and observe that omitted variable bias with respect to the DTD risk factor, neglect of interaction effects and incorrect link function specification has the greatest, intermediate and least impacts, respectively. We conclude that care must be taken to judiciously choose how we validate TTC vs. PIT models, as criteria may be rather different and apart from standards such as discriminatory power. This study contributes to the literature by providing expert guidance to credit risk modeling, model validation and supervisory practitioners in managing model risk.</p> </abstract>