External validation of a United Kingdom primary‐care Cushing's prediction tool in a population of referred Dutch dogs

Abstract

AbstractBackgroundA prediction tool was developed and internally validated to aid the diagnosis of Cushing's syndrome in dogs attending UK primary‐care practices. External validation is an important part of model validation to assess model performance when used in different populations.ObjectivesTo assess the original prediction model's transportability, applicability, and diagnostic performance in a secondary‐care practice in the Netherlands.AnimalsTwo hundred thirty client‐owned dogs.MethodsRetrospective observational study. Medical records of dogs under investigation of Cushing's syndrome between 2011 and 2020 were reviewed. Dogs diagnosed with Cushing's syndrome by the attending internists and fulfilling ALIVE criteria were defined as cases, others as non‐cases. All dogs were scored using the aforementioned prediction tool. Dog characteristics and predictor‐outcome effects in development and validation data sets were compared to assess model transportability. Calibration and discrimination were examined to assess model performance.ResultsEighty of 230 dogs were defined as cases. Significant differences in dog characteristics were found between UK primary‐care and Dutch secondary‐care populations. Not all predictors from the original model were confirmed to be significant predictors in the validation sample. The model systematically overestimated the probability of having Cushing's syndrome (a = −1.10, P < .001). Calibration slope was 1.35 and discrimination proved excellent (area under the receiver operating curve = 0.83).Conclusions and Clinical ImportanceThe prediction model had moderate transportability, excellent discriminatory ability, and overall overestimated probability of having Cushing's syndrome. This study confirms its utility, though emphasizes that ongoing validation efforts of disease prediction tools are a worthwhile effort.