Toward Lightweight Diabetic Retinopathy Classification: A Knowledge Distillation Approach for Resource-Constrained Settings

Abstract

Diabetic retinopathy (DR), a consequence of diabetes, is one of the prominent contributors to blindness. Effective intervention necessitates accurate classification of DR; this is a need that computer vision-based technologies address. However, using large-scale deep learning models for DR classification presents difficulties, especially when integrating them into devices with limited resources, particularly in places with poor technological infrastructure. In order to address this, our research presents a knowledge distillation-based approach, where we train a fusion model, composed of ResNet152V2 and Swin Transformer, as the teacher model. The knowledge learned from the heavy teacher model is transferred to the lightweight student model of 102 megabytes, which consists of Xception with a customized convolutional block attention module (CBAM). The system also integrates a four-stage image enhancement technique to improve the image quality. We compared the model against eight state-of-the-art classifiers on five evaluation metrics; the experiments show superior performance of the model over other methods on two datasets (APTOS and IDRiD). The model performed exceptionally well on the APTOS dataset, achieving 100% accuracy in binary classification and 99.04% accuracy in multi-class classification. On the IDRiD dataset, the results were 98.05% for binary classification accuracy and 94.17% for multi-class accuracy. The proposed approach shows promise for practical applications, enabling accessible DR assessment even in technologically underdeveloped environments.