Pre-trained convolution neural networks models for content-based medical image retrieval

Abstract

Content-based image retrieval (CBIR) is a recent method used to retrieve different types of images from repositories. The traditional content-based medical image retrieval (CBMIR) methods commonly used low-level image representation features extracted from color, texture, and shape image descriptors. Since most of these CBMIR systems depend mainly on the extracted features, the methods used in the feature extraction phase are more important. Features extraction methods, which generate inaccurate features, lead to very poor performance retrieval because of semantic gap widening. Hence, there is high demand for independent domain knowledge features extraction methods, which have automatic learning capabilities from input images. Pre-trained deep convolution neural networks (CNNs), the recent generation of deep learning neural networks, could be used to extract expressive and accurate features. The main advantage of these pre-trained CNNs models is the pre-training process for huge image data of thousands of different classes, and their knowledge after the training process could easily be transferred. There are many successful models of pre-trained CNNs models used in the area of medical image retrieval, image classification, and object recognition. This study utilizes two of the most known pre-trained CNNs models; ResNet18 and SqueezeNet for the offline feature extraction stage. Additionally, the highly accurate features extracted from medical images are used for the CBMIR method of medical image retrieval. This study uses two popular medical image datasets; Kvasir and PH2 to show that the proposed methods have good retrieval results. The retrieval performance evaluation measures of our proposed method have average precision of 97.75% and 83.33% for Kvasir and PH2 medical images respectively, and outperform some of the state-of-the-art methods in this field of study because these pre-trained CNNs have well trained layers among a huge number of image types. Finally, intensive statistical analysis shows that the proposed ResNet18-based retrieval method has the best performance for enhancing both recall and precision measures for both medical images.