Medial temporal cortex supports compositional visual inferences

Abstract

Perception unfolds across multiple timescales. For humans and other primates, many object-centric visual attributes can be inferred ‘at a glance’ (i.e., with<200ms of visual information), an ability supported by ventral temporal cortex (VTC). Other perceptual inferences require more time; to determine a novel object’s identity, we might need to represent its unique configuration of visual features, requiring multiple ‘glances.’ Here we evaluate whether medial temporal cortex (MTC), downstream from VTC, supports object perception by integrating over such visuospatial sequences. We first compare human visual inferences directly to electrophysiological recordings from macaque VTC. While human performance ‘at a glance’ is approximated by a linear readout of VTC, participants radically outperform VTC given longer viewing times (i.e.,>200ms). Next, we demonstrate the causal role of MTC in these temporally extended visual inferences: just as time restricted performance can be approximated by a linear readout of VTC, the performance of (time unrestricted) MTC-lesioned humans resembles a computational proxy for VTC. Finally, we characterize these visual abilities through a series of eyetracking experiments. With extended viewing times participants sequentially sample task-relevant features via multiple saccades—visuospatial patterns that are reliable across participants and necessary for performance. From these data, we suggest that MTC transforms visuospatial sequences into ‘compositional’ representations that support visual object perception.