The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset

Abstract

Introduction This article was undertaken to explore the potential of AI in enhancing the diagnostic accuracy and efficiency in identifying hip fractures using X-ray radiographs. In the study, we trained three distinct deep learning models, and we utilized majority voting to evaluate their outcomes, aiming to yield the most reliable and precise diagnoses of hip fractures from X-ray radiographs. Methods An initial study was conducted of 10,849 AP pelvis X-rays obtained from five hospitals affiliated with Başkent University. Two expert orthopedic surgeons initially labeled 2,291 radiographs as fractures and 8,558 as non-fractures. The algorithm was trained on 6,943 (64%) radiographs, validated on 1,736 (16%) radiographs, and tested on 2,170 (20%) radiographs, ensuring an even distribution of fracture presence, age, and gender. We employed three advanced deep learning architectures, Xception (Model A), EfficientNet (Model B), and NfNet (Model C), with a final decision aggregated through a majority voting technique (Model D). Results For each model, we achieved the following metrics: For Model A: F1 Score 0.895, Accuracy 0.956, Specificity 0.973, Sensitivity 0.893. For Model B: F1 Score 0.900, Accuracy 0.960, Specificity 0.991, Sensitivity 0.845. For Model C: F1 Score 0.919, Accuracy 0.966, Specificity 0.984, Sensitivity 0.899. For Model D: F1 Score 0.929, Accuracy 0.971, Specificity 0.991, Sensitivity 0.897. We concluded that Model D (majority voting) achieved the best results in terms of the F1 score, accuracy, and specificity values. Conclusions Our study demonstrates that the results obtained by aggregating the decisions of multiple models through voting, rather than relying solely on the decision of a single algorithm, are more consistent. The practical application of these algorithms will be difficult due to ethical, legal, and confidentiality issues, despite the theoretical success achieved. Developing successful algorithms and methodologies should not be viewed as the ultimate goal; it is important to understand how these algorithms will be used in real-life situations. In order to achieve more consistent results, feedback from clinical practice will be helpful.