Author:
Jia Shuncheng,Zhang Tielin,Zuo Ruichen,Xu Bo
Abstract
Network architectures and learning principles have been critical in developing complex cognitive capabilities in artificial neural networks (ANNs). Spiking neural networks (SNNs) are a subset of ANNs that incorporate additional biological features such as dynamic spiking neurons, biologically specified architectures, and efficient and useful paradigms. Here we focus more on network architectures in SNNs, such as the meta operator called 3-node network motifs, which is borrowed from the biological network. We proposed a Motif-topology improved SNN (M-SNN), which is further verified efficient in explaining key cognitive phenomenon such as the cocktail party effect (a typical noise-robust speech-recognition task) and McGurk effect (a typical multi-sensory integration task). For M-SNN, the Motif topology is obtained by integrating the spatial and temporal motifs. These spatial and temporal motifs are first generated from the pre-training of spatial (e.g., MNIST) and temporal (e.g., TIDigits) datasets, respectively, and then applied to the previously introduced two cognitive effect tasks. The experimental results showed a lower computational cost and higher accuracy and a better explanation of some key phenomena of these two effects, such as new concept generation and anti-background noise. This mesoscale network motifs topology has much room for the future.
Reference54 articles.
1. Metaplasticity: the plasticity of synaptic plasticity;Abraham;Trends Neurosci,1996
2. Slot machines: discovering winning combinations of random weights in neural networks,;Aladago,2021
3. Single cortical neurons as deep artificial neural networks;Beniaguev;Neuron,2021
4. Speaker-targeted audio-visual models for speech recognition in cocktail-party environments;Chao;arXiv [Preprint]. arXiv:1906.05962,2019
5. The lottery tickets hypothesis for supervised and self-supervised pre-training in computer vision models,;Chen,2020