Object-scene conceptual regularities reveal fundamental differences between biological and artificial object vision

Abstract

AbstractHuman vision is still largely unexplained. Computer vision made impressive progress on this front, but it is unclear to what extent artificial neural networks approximate human brain strategies. Here, we confirm this gap by testing how biological and artificial systems encode object-scene contextual regularities in natural images. Both systems represent these regularities, but the underlying information processing is markedly different. In human vision, objects and backgrounds are represented separately, with rich domain-specific representations characterizing human visual cortex. Interaction between these components occurs downstream in frontoparietal areas. Conversely, neural networks represent image components in a single entangled representation revealing reduced object-segregation abilities and impoverished domain-specific object spaces. These results show the uniqueness of human vision that allows understanding that images are not just a collection of features and points to the need for developing neural network models with a similar richness of representational content.