COVID-19 automatic diagnosis with CT images using the novel Transformer architecture

Abstract

Even though vaccines are already in use worldwide, the COVID-19 pandemic is far from over, with some countries re-establishing the lockdown state, the virus has taken over 2 million lives until today, being a serious health issue. Although real-time reverse transcription-polymerase chain reaction (RTPCR) is the first tool for COVID-19 diagnosis, its high false-negative rate and low sensitivity might delay accurate diagnosis. Therefore, fast COVID-19 diagnosis and quarantine, combined with effective vaccination plans, is crucial for the pandemic to be over as soon as possible. To that end, we propose an intelligent system to classify computed tomography (CT) of lung images between a normal, pneumonia caused by something other than the coronavirus or pneumonia caused by the coronavirus. This paper aims to evaluate a complete selfattention mechanism with a Transformer network to capture COVID-19 pattern over CT images. This approach has reached the state-of-the-art in multiple NLP problems and just recently is being applied for computer vision tasks. We combine vision transformer and performer (linear attention transformers), and also a modified vision transformer, reaching 96.00% accuracy.