Tools for Checking Calibration of a Cox Model in External Validation: Prediction of Population-Averaged Survival Curves Based on Risk Groups

Abstract

Royston (2014, Stata Journal 14: 738–755) explained how a popular application of the Cox proportional hazards model “is to develop a multivariable prediction model, often a prognostic model to predict the future clinical outcome of patients with a particular disorder from ‘baseline’ factors measured at some initial time point. For such a model to be useful in practice, it must be ‘validated’; that is, it must perform satisfactorily in an external sample of patients independent of the one on which the model was originally developed. One key aspect of performance is calibration, which is the accuracy of prediction, particularly of survival (or equivalently, failure or event) probabilities at any time after the time origin”. In this article, I suggest an approach to assess calibration by comparing observed (Kaplan–Meier) and predicted survival probabilities in several prognostic groups derived by placing cutpoints on the prognostic index. I distinguish between full validation, where all relevant quantities are estimated on the derivation dataset and predicted on the validation dataset, and partial validation, where the prognostic index and prognostic groups are derived from published information and the baseline distribution function is estimated in the validation dataset. Partial validation is more feasible in practice because it is uncommon to have access to individual patient values in both datasets. I exemplify the method by detailed analysis of two datasets in the disease primary biliary cirrhosis; the datasets comprise a derivation and a validation dataset. I describe a new ado-file, stcoxgrp, that performs the necessary calculations. Results for stcoxgrp are displayed graphically, which makes it easier for users to picture calibration (or lack thereof) according to follow-up time.