Convergence Analysis and Application for Multi‐Layer Neural Network Based on Fractional‐Order Gradient Descent Learning

Abstract

AbstractFractional order calculus, with its inheritance and infinite memory properties, is a promising research area in information processing and modeling of certain physical systems, system identification, and control. In this paper, a fractional‐order gradient descent method is proposed for backpropagation training of multilayer feedforward neural networks. In particular, the Caputo derivative is used to define the measurement function and consider the smooth regular term. In addition, the monotonicity of the error function and the strong (weak) convergence theorem of the algorithm are rigorously proved. Finally, numerical experiments prove the correctness and effectiveness of the algorithm theory.