Using unsupervised capsule neural network reveal spatial representations in the human brain

Abstract

AbstractHumans can extract high‐level spatial features from visual signals, but spatial representations in the brain are complex and remain unclear. The unsupervised capsule neural network (U‐CapsNet) is sensitive to the spatial location and relationship of the object, contains a special recurrent mechanism and uses a self‐supervised generation strategy to represent images, which is similar to the computational principle in the human brain. Therefore, we hypothesized that U‐CapsNet can help us understand how the human brain processes spatial information. First, brain activities were studied using functional magnetic resonance imaging during spatial working memory in which participants had to remember the locations of circles for a short time. Then, U‐CapsNet served as a computational model of the brain to perform tasks that are identical to those performed by humans. Finally, the representational models were used to compare the U‐CapsNet with the brain. The results showed that some human‐defined spatial features naturally emerged in the latent space of U‐CapsNet. Moreover, representations in U‐CapsNet captured the response structure of two types of brain regions during different activity patterns, as well as important factors associated with human behavior. Together, our study not only provides a computationally feasible framework for modeling how the human brain encodes spatial features but also provides insights into the representational format and goals of the human brain.