Diagnostic validation of vertebral heart score machine learning algorithm for canine lateral chest radiographs

Abstract

ObjectivesThe vertebral heart score is a measurement used to index heart size relative to thoracic vertebra. Vertebral heart score can be a useful tool for identifying and staging heart disease and providing prognostic information. The purpose of this study is to validate the use of a vertebral heart score algorithm compared to manual vertebral heart scoring by three board‐certified veterinary cardiologists.Materials and MethodsA convolutional neural network centred around semantic segmentation of relevant anatomical features was developed to predict heart size and vertebral bodies. These predictions were used to calculate the vertebral heart score. An external validation study consisting of 1200 canine lateral radiographs was randomly selected to match the underlying distribution of vertebral heart scores. Three American College of Veterinary Internal Medicine board‐certified cardiologists were enrolled to manually score 400 images each using the traditional Buchanan method. Post‐scoring, the cardiologists evaluated the algorithm for misaligned anatomic landmarks and overall image quality.ResultsThe 95th percentile absolute difference between the cardiologist vertebral heart score and the algorithm vertebral heart score was 1.05 vertebrae (95% confidence interval: 0.97 to 1.20 vertebrae) with a mean bias of −0.09 vertebrae (95% confidence interval: −0.12 to −0.05 vertebrae). In addition, the model was observed to be well calibrated across the predictive range.Clinical SignificanceWe have found the performance of the vertebral heart score algorithm comparable to three board‐certified cardiologists. While validation of this vertebral heart score algorithm has shown strong performance compared to veterinarians, further external validation in other clinical settings is warranted before use in those settings.