Abstract
AbstractDeep neural networks (DNN) have set new standards at predicting responses of neural populations to visual input. Most such DNNs consist of a convolutional network (core) shared across all neurons which learns a representation of neural computation in visual cortex and a neuron-specific readout that linearly combines the relevant features in this representation. The goal of this paper is to test whether such a representation is indeed generally characteristic for visual cortex, i.e. gener-alizes between animals of a species, and what factors contribute to obtaining such a generalizing core. To push all non-linear computations into the core where the generalizing cortical features should be learned, we devise a novel readout that reduces the number of parameters per neuron in the readout by up to two orders of magnitude compared to the previous state-of-the-art. It does so by taking advantage of retinotopy and learns a Gaussian distribution over the neuron’s receptive field po-sition. With this new readout we train our network on neural responses from mouse primary visual cortex (V1) and obtain a gain in performance of 7% compared to the previous state-of-the-art network. We then investigate whether the convolutional core indeed captures general cortical features by using the core in transfer learning to a different animal. When transferring a core trained on thousands of neurons from various animals and scans we exceed the performance of training directly on that animal by 12%, and outperform a commonly used VGG16 core pre-trained on imagenet by 33%. In addition, transfer learning with our data-driven core is more data-efficient than direct training, achieving the same performance with only 40% of the data. Our model with its novel readout thus sets a new state-of-the-art for neural response prediction in mouse visual cortex from natural images, generalizes between animals, and captures better characteristic cortical features than current task-driven pre-training approaches such as VGG16.
Publisher
Cold Spring Harbor Laboratory
Reference47 articles.
1. Model Constrained by Visual Hierarchy Improves Prediction of Neural Responses to Natural Scenes
2. Neural Population Control via Deep ANN Image Synthesis
3. E. Batty , J. Merel , N. Brackbill , A. Heitman , A. Sher , A. Litke , E. J. Chichilnisky , and L. Paninski . Multilayer network models of primate retinal ganglion cells. Number Nips, 2016.
4. Deep convolutional models improve predictions of macaque V1 responses to natural images
5. S. A. Cadena , F.H. Sinz , T. Muhammad , E. Froudarakis , E. Cobos , E. Y. Walker , J. Reimer , M. Bethge , and A. S. Ecker . How well do deep neural networks trained on object recognition characterize the mouse visual system? In NeurIPS 2019 Workshop Neuro AI, 2019b.
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献