Genetic-efficient fine-tuning with layer pruning on multimodal Covid-19 medical imaging

Abstract

AbstractMedical image analysis using multiple modalities refers to the process of analyzing and extracting information from more than one type of image in order to gain a comprehensive understanding of a given subject. To maximize the potential of multimodal data in improving and enhancing our understanding of the disease, sophisticated classification techniques must be developed as part of the integration process to classify meaningful information from different types of data. A pre-trained model, such as those trained on large datasets such as ImageNet, has learned rich representations that can be used for various downstream tasks. Fine-tuning a pre-trained model refers to the process of further developing the model using the knowledge and representations gained from a pre-existing dataset. In comparison to training a model from scratch, fine-tuning allows knowledge to be transferred from the pre-trained model to the target task, thus improving performance and efficiency. In evolutionary search, the genetic algorithm (GA) is an algorithm that emulates the process of natural selection and genetics. In this context, a population of candidate solutions is generated, fitness is evaluated and new candidate solutions are generated by applying genetic operations such as mutation and crossover. Considering the above characteristics, the present study presents an efficient architecture called Selective-COVIDNet for analyzing COVID-19 cases using a novel selective layer-pruning algorithm. To detect COVID-19 from multimodal data, the current study will use a genetic algorithm to fine-tune the performance of pre-trained models by adjusting specific layers selectively. Furthermore, the proposed approach provides flexibility in the depth of two deep learning architectures, VGG-16 and MobileNet-V2. The impact of freezing specific layers on fine-tuning performance was assessed using five different strategies, namely Random, Odd, Even, Half, and Full Freezing. Therefore, existing pre-trained models can be enhanced for Covid-19 tasks while minimizing their computational burden. For evaluating the effectiveness of the proposed framework, two multi-modal standard datasets are used, including CT-scan images and electrocardiogram (ECG) recordings of individuals with COVID-19. From the conducted experiments, it is found that the proposed framework can detect Covid-19 effectively with accuracy of 98.48% for MobileNet-V2 and 99.65% for VGG-16.