Towards implementation of AI in New Zealand national screening program: Cloud-based, Robust, and Bespoke

Abstract

AbstractConvolutional Neural Networks (CNN)s have become a prominent method of AI implementation in medical classification tasks. Grading Diabetic Retinopathy (DR) has been at the forefront of the development of AI for ophthalmology. However, major obstacles remain in the generalization of these CNN’s onto real-world DR screening programs. We believe these difficulties are due to use of 1) small training datasets (<5,000 images), 2) private and ‘curated’ repositories, 3) offline CNN implementation methods, while 4) relying on accuracy measured as area under the curve (AUC) as the sole measure of CNN performance.To address these issues, the public EyePACS Kaggle Diabetic Retinopathy dataset was uploaded onto Microsoft Azure™ cloud platform. Two CNNs were trained as a “Quality Assurance”, and a “Classifier”. The “Classifier” CNN performance was then tested both on ‘un-curated’ as well as the ‘curated’ test set created by the “Quality Assessment” CNN. Finally, the sensitivity of the “Classifier” CNNs was boosted post-training using two post-training techniques.Our “Classifier” CNN proved to be robust, as its performance was similar on ‘curated’ and ‘uncurated’ sets. The implementation of ‘cascading thresholds’ and ‘max margin’ techniques led to significant improvements in the “Classifier” CNN’s sensitivity, while also enhancing the specificity of other grades.