Thorax computed tomography (CTX) guided ground truth annotation of CHEST radiographs (CXR) for improved classification and detection of COVID‐19

Abstract

AbstractSeveral data sets have been collected and various artificial intelligence models have been developed for COVID‐19 classification and detection from both chest radiography (CXR) and thorax computed tomography (CTX) images. However, the pitfalls and shortcomings of these systems significantly limit their clinical use. In this respect, improving the weaknesses of advanced models can be very effective besides developing new ones. The inability to diagnose ground‐glass opacities by conventional CXR has limited the use of this modality in the diagnostic work‐up of COVID‐19. In our study, we investigated whether we could increase the diagnostic efficiency by collecting a novel CXR data set, which contains pneumonic regions that are not visible to the experts and can only be annotated under CTX guidance. We develop an ensemble methodology of well‐established deep CXR models for this new data set and develop a machine learning‐based non‐maximum suppression strategy to boost the performance for challenging CXR images. CTX and CXR images of 379 patients who applied to our hospital with suspected COVID‐19 were evaluated with consensus by seven radiologists. Among these, CXR images of 161 patients who also have had a CTX examination on the same day or until the day before or after and whose CTX findings are compatible with COVID‐19 pneumonia, are selected for annotating. CTX images are arranged in the main section passing through the anterior, middle, and posterior according to the sagittal plane with the reformed maximum intensity projection (MIP) method in the coronal plane. Based on the analysis of coronal MIP reconstructed CTX images, the regions corresponding to the pneumonia foci are annotated manually in CXR images. Radiologically classified posterior to anterior (PA) CXR of 218 patients with negative thorax CTX imaging were classified as COVID‐19 pneumonia negative group. Accordingly, we have collected a new data set using anonymized CXR (JPEG) and CT (DICOM) images, where the PA CXRs contain pneumonic regions that are hidden or not easily recognized and annotated under CTX guidance. The reference finding was the presence of pneumonic infiltration consistent with COVID‐19 on chest CTX examination. COVID‐Net, a specially designed convolutional neural network, was used to detect cases of COVID‐19 among CXRs. Diagnostic performances were evaluated by ROC analysis by applying six COVID‐Net variants (COVIDNet‐CXR3‐A, ‐B, ‐C/COVIDNet‐CXR4‐A, ‐B, ‐C) to the defined data set and combining these models in various ways via ensemble strategies. Finally, a convex optimization strategy is carried out to find the outperforming weighted ensemble of individual models. The mean age of 161 patients with pneumonia was 49.31 ± 15.12, and the median age was 48 years. The mean age of 218 patients without signs of pneumonia in thorax CTX examination was 40.04 ± 14.46, and the median was 38. When working with different combinations of COVID‐Net's six variants, the area under the curve (AUC) using the ensemble COVID‐Net CXR 4A‐4B‐3C was .78, sensitivity 67%, specificity 95%; COVID‐Net CXR 4a‐3b‐3c was .79, sensitivity 69% and specificity 94%. When diverse and complementary COVID‐Net models are used together through an ensemble, it has been determined that the AUC values are close to other studies, and the specificity is significantly higher than other studies in the literature.