Information Processing during Face Recognition: The Effects of Familiarity, Inversion, and Morphing on Scanning Fixations

Abstract

Where we make ocular fixations when viewing an object likely reflects interactions between ‘external’ object properties and internal ‘top - down’ factors, as our perceptual system tests hypotheses and attempts to make decisions about our environment. These scanning fixation patterns can tell us how and where the visual system gathers information critical to specific tasks. We determined the effects of the internal factors of expertise, experience, and ambiguity on scanning during a face-recognition task, in eight subjects. To assess the effects of expertise, we compared upright with inverted faces, since it is hypothesized that inverted faces do not access an orientation-dependent face-expert processor. To assess the effects of experience, we compared famous with novel faces, as famous faces would have stronger internal representations than anonymous ones. Ambiguity in matching seen and remembered faces was manipulated with morphed faces. We measured three classes of variables: (i) total scanning time and fixations; (ii) the spatial distribution of scanning; and (iii) the sequence of scanning, using first-order Markov matrices for local scan structure and string editing for global scan structure. We found that, with inverted faces, subjects redistributed fixations to the mouth and lower face, and their local and global scan structure became more random. With novel or morphed faces, they scanned the eyes and upper face more. Local scan structure was not affected by familiarity, but global scan structure was least random (most stereotyped) for novel upright faces. We conclude that expertise (upright faces) leads to less lower-face scanning and more predictable global patterns of information gathering. Experience (famous faces) leads to less upper-face scanning and more idiosyncratic global scan structures, suggesting a superseding influence of facial memories. With morphed faces, subjects return to the upper face to resolve ambiguity, implying a greater importance of this region in face recognition.