Accuracy of an artificial intelligence-based mobile application for detecting cataracts: Results from a field study

Abstract

Purpose: To assess the accuracy of e-Paarvai, an artificial intelligence-based smartphone application (app) that detects and grades cataracts using images taken with a smartphone by comparing with slit lamp-based diagnoses by trained ophthalmologists. Methods: In this prospective diagnostic study conducted between January and April 2022 at a large tertiary-care eye hospital in South India, two screeners were trained to use the app. Patients aged >40 years and with a best-corrected visual acuity <20/40 were recruited for the study. The app is intended to determine whether the eye has immature cataract, mature cataract, posterior chamber intra-ocular lens, or no cataract. The diagnosis of the app was compared with that of trained ophthalmologists based on slit-lamp examinations, the gold standard, and a receiver operating characteristic (ROC) curve was estimated. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were computed. Results: The two screeners used the app to screen 2,619 eyes of 1,407 patients. In detecting cataracts, the app showed high sensitivity (96%) but low specificity (25%), an overall accuracy of 88%, a PPV of 92.3%, and an NPV of 57.8%. In terms of cataract grading, the accuracy of the app was high in detecting immature cataracts (1,875 eyes, 94.2%), but its accuracy was poor in detecting mature cataracts (73 eyes, 22%), posterior chamber intra-ocular lenses (55 eyes, 29.3%), and clear lenses (2 eyes, 2%). We found that the area under the curve in predicting ophthalmologists’ cataract diagnosis could potentially be improved beyond the app’s diagnosis based on using images only by incorporating information about patient sex and age (P < 0.0001) and best-corrected visual acuity (P < 0.0001). Conclusions: Although there is room for improvement, e-Paarvai app is a promising approach for diagnosing cataracts in difficult-to-reach populations. Integrating this with existing outreach programs can enhance the case detection rate.