Deformable 3D medical image registration with convolutional neural network and transformer

Abstract

Abstract Deformable registration of medical images based on deep learning has been the research focus this year. Convolutional Neural Network (CNN) and the transformer are the most common backbone and have been shown to enhance registration accuracy. However, CNN lacks the ability to contact long-distance information, and the transformer lacks the ability to capture local information. Whichever subtle feature loss may lead to disastrous consequences in the analysis of clinical medicine. This paper presented a novel registration network named Information Complementation Network (ICN). We aim to improve the registration accuracy by complementing the lost information. Pure transformers can establish long-distance spatial information about the image. Proposed meshing patch embedding can minimize the loss of local information and expand the receptive field to extract long-distance information. The dual-path decoder in ICN is designed to restore information furthest. We experimented on 3D brain MRI data and quantitatively compared several excellent registration models. Compared with conventional methods, the dice coefficient increased by 3%. Compared with the advanced methods, the dice coefficient increased by 1%. The number of foldings was reduced by about 50% without any loss of registration accuracy. Each evaluation metric of the trained models on liver CT images was higher than other methods. By fully complementing the lost or invalid information, ICN achieved higher registration accuracy and smoother deformation field. The innovation and contribution of this paper have the potential to be applied to clinical research or medical image processing.