Leveraging non-expert crowdsourcing to segment the optic cup and disc of multicolor fundus images

Abstract

Multicolor scanning laser imaging (MCI) images have broad application potential in the diagnosis of fundus diseases such as glaucoma. However, the performance level of automatic aided diagnosis systems based on MCI images is limited by the lack of high-quality annotations of numerous images. Producing annotations for vast amounts of MCI images will be a prolonged process if we only employ experts. Therefore, we consider non-expert crowdsourcing, which is an alternative approach to produce useful annotations efficiently and low cost. In this work, we aim to explore the effectiveness of non-expert crowdsourcing on the segmentation of the optic cup (OC) and optic disc (OD), which is an upstream task for glaucoma diagnosis, using MCI images. To this end, desensitized MCI images are independently annotated by four non-expert annotators, constructing a crowdsourcing dataset. To profit from crowdsourcing, we propose a model consisting of coupled regularization network and segmentation network. The regularization network generates learnable pixel-wise confusion matrices (CMs) that reflects preferences of each annotator. During training, the CMs and segmentation network are simultaneously optimized to enable dynamic trade-offs for non-expert annotations and generate reliable predictions. Crowdsourcing learning using our method have an average Mean Intersection Over Union (M) of 91.34%, while the averageMof model trained by expert annotations is 91.72%. In addition, comparative experiments show that in our segmentation task non-expert crowdsourcing can be on a par with the expert who annotates 90% of data. Our work suggests that crowdsourcing in the segmentation of OC and OD using MCI images has the potential to be a substitute to expert annotation, which will accelerate the construction of large datasets to facilitate the application of deep learning in clinical diagnosis using MCI images.