Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Abstract

AbstractEfficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. Distinguishing self-from externally caused changes in visual input is thus an important problem that the visual system needs to solve. Predictive coding with sensorimotor mismatch detection is an attractive starting point to investigate this question computationally. Although experimental evidence for sensorimotor mismatch signals in early visual areas exists, it is not understood how it is functionally integrated into cortical networks that perform input segmentation and categorization. Our model advanced a novel, biologically plausible solution to this question, which extends predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron microcircuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a predictive coding neural network with two generative streams. The first stream is motor-to-visual and consists of many microcircuits in parallel. This stream learns to spatially predict optic flow resulting from self-motion and mirrors connections from motor cortex to V1. The second stream is visual-to-visual. Bidirectionally connecting Middle Temporal cortex to V1, it assigns a crucial role to the abundant feedback connections between these areas: the maintenance of a generative model of externally caused optic flow. In the model, area MT learns to segment moving objects from the background, and facilitates object categorization. Our model extends the framework of Hebbian predictive coding to sensorimotor settings, in which the agent is not a passive observer of external inputs, but actively moves - and learns to predict the consequences of its own movements.Significance statementThis research addresses a fundamental challenge in sensory perception: how the brain distinguishes between self-generated and externally caused visual motion. Using a computational model inspired by predictive coding and sensorimotor mismatch detection, the study proposes a biologically plausible solution. The model incorporates a neural microcircuit that generates sensorimotor mismatch responses, aligning with experimental data from mice. This microcircuit is integrated into a neural network with two streams: one predicting self-motion-induced optic flow and another maintaining a generative model for externally caused optic flow. The research advances our understanding of how the brain segments visual input into object and background, shedding light on the neural mechanisms underlying perception and categorization.