How the brain learns to parse images using an attentional, incremental grouping process

Abstract

AbstractNatural scenes usually contain a vast number of objects that need to be segmented and segregated from each other and from the background to guide behaviour. In the visual brain, object-based attention is the process by which image fragments belonging to the same objects are grouped together. The curve-tracing task is a special case of a perceptual grouping task that tests our ability to group image elements of an elongated curve. The task consists in determining which image elements belong to the same curve, and in the brain, neurons spread an enhanced activity level over the representation of the relevant curve. A previous “growth-cone model of attention” accounted for the scale invariance of tracing by proposing that the enhanced activity propagates at multiple levels of the visual cortical hierarchy. However, the precise neuronal circuitry for learning and implementing scale-invariant tracing remains unknown. We propose a new recurrent architecture for the scale-invariant labelling of curves and objects. The architecture is composed of a feedforward pathway that dynamically selects the right scale and prevents the spilling over of the enhanced activity to other curves, and a recurrent pathway for tag spreading that involves horizontal and feedback interactions, mediated by a disinhibitory loop involving VIP and SOM interneurons. We trained the network with curves up to seven pixels long using reinforcement learning and a learning rule local in time and space and we found that it generalized to curves of any length and to spatially extended objects. The network chose the appropriate scale and switched to higher or lower scales as dictated by the distance between curves, just has as been observed in human psychophysics and in the visual cortex of monkeys. Our work provide a mechanistic account of the learning of scale-invariant perceptual grouping in the brain.Significance StatementObjects are labelled and grouped in the visual cortex via a tag of enhanced activity. If the scale-invariant dynamics of propagations of this tag are well characterised, it remains unknown what neural architectures and learning rules can produce those dynamics. This work is the first to propose a neural architecture trained with reward that give rises to the same dynamics observed in monkeys’ visual cortex or human reaction times, shedding light on the mechanisms of multiscale object-based attention in the visual cortex.