Research on Approximate Fitting Method of Activation Function Based on Gradient Equalization in Deep Learning

Abstract

Abstract Activation function plays an important role in neural network model. Neural networks can express more complex situations by introducing nonlinearity due to activation functions. In this paper, an approximate fitting method of activation function based on gradient bisection is proposed. The algorithm can fit the activation function curve well. You can use less space for storage on your computer. Through experiments, the storage capacity can be reduced by one time compared with the traditional approximate fitting method, but the accuracy of the model is not affected.