A novel approach of visual image reconstruction from fMRI using Kohonen Network Information Maximizing Generative Adversarial Network

Abstract

As neuroscience states, the ability of our brain to process visual information is similar to that of neurons. Our cognitive ability carries out this process using the functional magnetic resonance imaging (fMRI) profiles that it collects during the presentation of visual stimuli around us. However, this process poses its own set of challenges due to complexities like structure and noise levels. To address these challenges, high-quality computational approaches have been executed. This work proposes a hybrid computational approach, that combines feature extraction and conditional generative adversarial network techniques. This research suggests the use of the hybrid form incorporating Kohonen Network and Information Maximizing Generative Adversarial Networks (KNInfoGAN) to map the fMRI data to visual stimuli. For feature extraction, the Kohonen network is used, and the visual stimuli are reconstructed using the InfoGAN approach. A series of experiments were conducted on dual synthetic fMRI datasets using a KNInfoGAN model, and the results were obtained with an accuracy rate of 99% for handwritten numbers and 87% for handwritten English characters, approximately. An accuracy rate of 73% has been obtained upon utilizing the ImageNet database to examine the accomplishments of the suggested approach for highly complex natural stimulus. The suggested technique in this study outperforms the previously existing techniques in image reconstruction, with enhanced quality and a lower computational cost. The same has been verified based on the comparative examination of a wide range of tests conducted in this study.