A double-blinded study for quantifiable assessment of the diagnostic accuracy of AI tool “ADVEN-i” in identifying diseased fundus images including diabetic retinopathy on a retrospective data

Abstract

Purpose: To quantifiably assess the diagnostic accuracy of Adven-I, a proprietary artificial intelligence (AI)-driven diagnostic system that automatically detects diseases from fundus images. The purpose is to quantify the performance of Adven-i in differentiating a nonreferable (within normal limits) image from a referable (diseased fundus) image and further segregating diabetic retinopathy (DR) from the rest of the abnormalities (non-DR) encompassing the wide spectrum of abnormal pathologies. The assessment is carried out in comparison to manual reading as the reference gold standard. Adven-i is the only AI system classifying retinal abnormalities into DR and non-DR classes separately, apart from predicting nonreferable fundus, while most existing systems classify fundus images into referable and nonreferable DR. Methods: The double-blinded study was conducted on retrospective data collected over the course of a year in the ophthalmology outpatient department (OPD) at a top Tier II eyecare hospital in Chandigarh, India. Three vitreoretina specialists who were blinded to one another read the images. The ground-truth was generated on the basis of majority agreement among the readers. An arbitrator's decision was regarded final if all three readers disagreed. Results: 2261 fundus images were analyzed by Adven-i. The sensitivity and specificity of Adven-i in diagnosing images with abnormalities were 95.12% and 85.77%, respectively, and for segregating DR from rest of the retinal abnormalities were 91.87% and 85.12%, respectively. Conclusions and Relevance: Adven-i shows definite promise in automated screening for early diagnosis of referable fundus images including DR. Adven-i can be adopted to scale for mass screening in resource-limited settings.