Affiliation:
1. Institute of Sound and Vibration Research, University of Southampton , Southampton SO17 1BJ, United Kingdom
Abstract
A paper by the current authors Paul and Nelson [JASA Express Lett. 3(9), 094802 (2023)] showed how the singular value decomposition (SVD) of the matrix of real weights in a neural network could be used to prune the network during training. The paper presented here shows that a similar approach can be used to reduce the training time and increase the implementation efficiency of complex-valued neural networks. Such networks have potential advantages compared to their real-valued counterparts, especially when the complex representation of the data is important, which is the often case in acoustic signal processing. In comparing the performance of networks having both real and complex elements, it is demonstrated that there are some advantages to the use of complex networks in the cases considered. The paper includes a derivation of the backpropagation algorithm, in matrix form, for training a complex-valued multilayer perceptron with an arbitrary number of layers. The matrix-based analysis enables the application of the SVD to the complex weight matrices in the network. The SVD-based pruning technique is applied to the problem of the classification of transient acoustic signals. It is shown how training times can be reduced, and implementation efficiency increased, while ensuring that such signals can be classified with remarkable accuracy.
Funder
Engineering and Physical Sciences Research Council
Publisher
Acoustical Society of America (ASA)
Reference45 articles.
1. Complex-valued signal processing: The proper way to deal with impropriety;IEEE Trans. Signal Process.,2011
2. Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks,2011
3. Pruning algorithms of neural networks: A comparative study;Open Comput. Sci.,2013
4. Bassey,
J.,
Qian,
L., and
Li,
X. (2021). “
A survey of complex-valued neural networks,” arXiv:2101.12249.
5. On the complex backpropagation algorithm;IEEE Trans. Signal Process.,1992