A cross-attention based image fusion Network for prediction of mild cognitive impairment

Abstract

Abstract Mild cognitive impairment (MCI) is the intermediate stage in the progression of Alzheimer’s disease, where patients exhibit cognitive decline, decreasing ability to perform complex daily tasks and other symptoms that can be mistaken for normal aging, thus missing the optimal time for treatment and leading to further deterioration of the patient’s condition. Structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) are commonly used to examine patients in clinical practice. Currently, the fusion of sMRI and PET based on deep learning are simply extracting features individually and then fusing the extracted features, ignoring the correlation between features of two different modalities. Therefore, this paper proposes an image fusion method based on cross-attention mechanism, which makes it possible to focus not only on the features of its own view but also on the features of the other view during feature extraction to achieve dual-attempt cross-learning. The experimental results from the Alzheimer’s Disease Neuroimaging Initiative dataset show that the images fused based on the method proposed in this paper outperform the original method in three evaluation indexes: peak signal-to-noise ratio, structural similarity and classification accuracy.