The role of binocular disparity in the neural representation of multiple moving stimuli in the visual cortex

Abstract

AbstractA fundamental process of vision is to segment visual scenes into distinct objects and surfaces. Stereoscopic depth and visual motion cues are particularly important for segmentation. However, how the primate visual system uses depth and motion cues to segment multiple surfaces in 3-dimensional space is not well understood. We investigated how neurons in the middle temporal (MT) cortex represented two overlapping surfaces located at different depths and moved simultaneously in different directions. We recorded neuronal activities in MT of three male macaque monkeys while they performed discrimination tasks under different attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the horizontal disparity of one of the two surfaces. For all animals, the disparity bias in response to two surfaces was positively correlated with the disparity preference of the neurons to single surfaces. For two animals, neurons that preferred the near disparities of single surfaces (near neurons) showed a near bias to overlapping stimuli, and neurons that preferred the far disparities (far neurons) showed a far bias. For the third animal, both near and far neurons showed a near bias, although the near neurons showed a stronger near bias than the far neurons. Interestingly, for all three animals, both near and far neurons showed an initial near bias relative to the average of the responses to the individual surfaces. Although attention can modulate neuronal response to better represent the attended surface, the disparity bias was still present when attention was directed away from the visual stimuli, indicating that the disparity bias cannot be explained by an attention bias. We also found that the effect of attention modulation of MT responses was consistent with object-based rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. Our model is a novel extension of the standard normalization model and provides a unified explanation of the disparity bias across animals. Our results revealed the neural encoding rule for multiple moving stimuli located at different depths and showed new evidence of response modulation by object-based attention in MT. The disparity bias would allow subgroups of neurons to preferentially represent individual surfaces at different depths of multiple stimuli and therefore facilitate segmentation. Attention can further select a surface and enhance its neural representation.