Ethnicity is not biology: retinal pigment score to evaluate biological variability from ophthalmic imaging using machine learning

Abstract

AbstractBackgroundFew metrics exist to describe phenotypic diversity within ophthalmic imaging datasets, with researchers often using ethnicity as an inappropriate marker for biological variability.MethodsWe derived a continuous, measured metric, the retinal pigment score (RPS), that quantifies the degree of pigmentation from a colour fundus photograph of the eye. RPS was validated using two large epidemiological studies with demographic and genetic data (UK Biobank and EPIC-Norfolk Study).FindingsA genome-wide association study (GWAS) of RPS from UK Biobank identified 20 loci with known associations with skin, iris and hair pigmentation, of which 8 were replicated in the EPIC-Norfolk cohort. There was a strong association between RPS and ethnicity, however, there was substantial overlap between each ethnicity and the respective distributions of RPS scores.InterpretationRPS serves to decouple traditional demographic variables, such as ethnicity, from clinical imaging characteristics. RPS may serve as a useful metric to quantify the diversity of the training, validation, and testing datasets used in the development of AI algorithms to ensure adequate inclusion and explainability of the model performance, critical in evaluating all currently deployed AI models. The code to derive RPS is publicly available at:https://github.com/uw-biomedical-ml/retinal-pigmentation-score.FundingThe authors did not receive support from any organisation for the submitted work.Research in contextEvidence before this studyVision loss due to retinal disease is a global problem as populations age and diabetes becomes increasingly prevalent. AI algorithms developed for efficient diagnosis of diabetic retinopathy and age-related macular degeneration rely on large imaging datasets collected from clinical practice. A substantial proportion (more than 80%) of publicly available retinal imaging datasets lack data on participant demographic characteristics. Some ethnic groups are noticeably underrepresented in medical research.Previous findings in dermatology suggest that AI algorithms can show reduced performance on darker skin tones. Similar biases may exist in retinal imaging, where retinal colour has been shown to affect disease detection.Added value of this studyWe introduce the Retinal Pigment Score (RPS), a measure of retinal pigmentation from digital fundus photographs. This score showed strong, reproducible associations with genetic variants related to skin, eye, and hair colour. Additionally, we identify three genetic loci potentially unique to retinal pigmentation, which warrant further investigation. The RPS provides an accurate and objective metric to describe the biological variability of the retina directly derived from an image.Implications of all the available evidenceThe RPS method represents a valuable metric with importance to harness the detailed information of ophthalmic fundus imaging. Its application implies potential benefits, such as improved accuracy and inclusivity, over human-created sociodemographic classifications used in dataset compilation and in the processes of developing and validating models. The RPS could decouple the distinct social and political categorical constructs of race and ethnicity from image analysis. It is poised to both accurately describe the diversity of a population study dataset or an algorithm training dataset, and for investigate algorithmic bias by assessing outcomes.Further work is needed to characterise RPS across different populations, considering individual ocular factors and different camera types. The development of standard reporting practices using RPS for studies employing colour fundus photography is also critical.