CarpNet: Transformer for mitral valve disease classification in echocardiographic videos

Abstract

AbstractMitral valve (MV) diseases constitute one of the etiologies of cardiovascular mortality and morbidity. MV pathologies need evaluating and classifying via echocardiographic videos. Transformers have significantly advanced video analytics. MV motion is divided by Carpentier functional classification into four types: normal, increased, restricted, and restricted only during systole. This paper introduces CarpNet, a deep transformer network that incorporates video transformers capable of direct MV pathology Carpentier's classification from the parasternal long‐axis (PLA) echocardiographic videos. The network, instead of processing frames independently, analyzes stacks of temporally consecutive frames using multi‐head attention modules to incorporate MV temporal dynamics into the learned model. To that end, different convolutional neural networks (CNNs) are evaluated as the backbone, and the best model is selected using the information of the PLA view. The use of information obtained by our proposed deep transformer network from consecutive echocardiographic frames yielded better results concerning the Carpentier functional classification than information obtained by CNN‐based (single‐frame) models. Using the Inception_Resnet_V2 architecture as the backbone, CarpNet achieved 71% accuracy in the test dataset. Deep learning and transformers in echocardiographic videos can render quick, precise, and stable evaluations of various MV pathologies.