Accuracy and reliability of automated three-dimensional facial landmarking in medical and biological studies. A systematic review

Abstract

Summary Background 3D facial landmarking is becoming a fundamental part of clinical and biological applications. Manual landmarking is time consuming and prone to cumulative errors, so attempts have been made to automate 3D facial landmarking. However, data in the literature are sparse. Objectives The objectives of this study are to investigate current evidence for the accuracy and reliability of various 3D facial automated landmarking methods used in medical and biological studies and evaluate their performance against the manual annotation method. Search methods Electronic and manual searches of the literature were performed in April 2021. Selection criteria Only studies that were published in English and evaluated the accuracy of automated landmarking algorithms in 3D facial images for medical or biological settings were included. Data collection and analysis Two authors independently screened the articles for eligibility. The QUADAS-2 tool was used for the quality analysis of the included studies. Due to the heterogeneity of the selected studies, a meta-analysis was not possible, so a narrative synthesis of the findings was performed. Results From 1002 identified records, after applying the inclusion and exclusion criteria, 14 articles were ultimately selected, read, and critically analysed. Different algorithms were used for the automated 3D landmarking of various numbers of facial landmarks ranging from 10 to 29 landmarks. The average difference between the manual and automated methods ranged from 0.67 to 4.73 mm, and the best performance was achieved using deep learning models. Poor study design and inadequate reporting were found in the implementation of the reference standards and population selection for the intended studies, which could have led to overfitting of the tested algorithm. Limitations This systematic review was limited by the quality of the included studies and uncovered several methodological limitations evident in the corresponding literature Conclusion and implications Compared to manual landmarking, automated Landmark localization of individual facial landmarks reported in the literature is not accurate enough to allow their use for clinical purposes. This result indicates that automatic facial landmarking is still developing, and further studies are required to develop a system that could match or exceed the performance of the current gold standard. Registration PROSPERO: CRD42021241531.