Distributed Reduced Convolution Neural Networks

Abstract

The fields of pattern recognition and machine learning frequently make use of something called a Convolution Neural Network, or CNN for short. The kernel extension of CNN, often known as KCNN, offers a performance that is superior to that of conventional CNN. When working with a large-size kernel matrix, the KCNN takes a lot of time and requires a lot of memory, despite the fact that it is capable of solving difficult nonlinear problems. The implementation of a reduced kernel approach has the potential to significantly lower the amount of computational burden and memory consumption. However, since the total quantity of training data continues to expand at an exponential rate, it becomes impossible for a single worker to store the kernel matrix in an efficient manner. This renders centralized data mining impossible to implement. A distributed reduced kernel approach for training CNN on decentralized data, which is referred to as DRCNN, is proposed in this study. In the DRCNN, we will arbitrarily distribute the data to the various nodes. The communication between nodes is static and does not depend on the amount of training data stored on each node; instead, it is determined by the architecture of the network. In contrast to the reduced kernel CNN that is already in use, the DRCNN is a completely distributed training technique that is based on the approach of alternating direction multiplier (ADMM). Experimentation with the large size data set reveals that the distributed technique can produce virtually the same outcomes as the centralized algorithm, and it even requires less time to a significant amount. It results in a significant decrease in the amount of time needed for computation.