Gray Matters: An Efficient Vision Transformer GAN Framework for Predicting Functional Network Connectivity Biomarkers from Brain Structure

Abstract

AbstractThe field of brain connectivity research has under-gone revolutionary changes thanks to state-of-the-art advancements in neuroimaging, particularly regarding structural and functional magnetic resonance imaging (MRI). To navigate the intricate neural dynamics, one must possess a keen comprehension of the interdependent links between structure and function. Such relationships are understudied as they are complex and likely nonlinear. To address this, we created a new generative deep learning architecture using a conditional efficient vision transformer generative adversarial network (cEViTGAN) to capture the distinct information in structural and functional MRI of the human brain. Our model generates functional network connectivity (FNC) matrices directly from three-dimensional sMRI data. Two pioneering innovations are central to our approach. First, we use a novel linear embedding method for structural MRI (sMRI) data that retains the 3D spatial detail. This embedding is best for representative learning, and when used on a consistent dataset, and shows that it is good at upstream classification assignments. To estimate neural biomarkers, we need to process much smaller patches using ViT-based architectures, which usually makes the computations more difficult because of the self-attention operations. We present a new, lightweight self-attention mechanism to address this challenge. Our mechanism not only overcomes computational shortcomings of traditional softmax self-attention but also surpasses pure linear self-attention models in accuracy and performance. This optimization enables us to analyze even the tiniest neuroanatomical details with exceptional precision. Our model allows for the identification of functional network connectivity (FNC) with 74.2% accuracy and also predicts subject differences in FNC for schizophrenia patients versus controls. The results are intriguing and suggest the links between gray matter volume and brain function may be stronger than previously considered.