Abstract
SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.
Publisher
Cold Spring Harbor Laboratory
Reference62 articles.
1. Neural population control via deep image synthesis
2. Cadena, S.A. , Sinz, F.H. , Muhammad, T. , Froudarakis, E. , Cobos, E. , Walker, E.Y. , Reimer, J. , Bethge, M. , Tolias, A.S. , and Ecker, A.S. (2019). How well do deep neural networks trained on object recognition characterize the mouse visual system? 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. https://openreview.net/forum?id=rkxcXmtUUS
3. Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
4. Spatial coding and invariance in object-selective cortex
5. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence