Development of an Efficient Variable Step-Size Gradient Method Utilizing Variable Fractional Derivatives

Abstract

The fractional gradient method has garnered significant attention from researchers. The common view regarding fractional-order gradient methods is that they have a faster convergence rate compared to classical gradient methods. However, through conducting theoretical convergence analysis, we have revealed that the maximum convergence rate of the fractional-order gradient method is the same as that of the classical gradient method. This discovery implies that the superiority of fractional gradients may not reside in achieving fast convergence rates compared to the classical gradient method. Building upon this discovery, a novel variable fractional-type gradient method is proposed with an emphasis on automatically adjusting the step size. Theoretical analysis confirms the convergence of the proposed method. Numerical experiments demonstrate that the proposed method can converge to the extremum point both rapidly and accurately. Additionally, the Armijo criterion is introduced to ensure that the proposed gradient methods, along with various existing gradient methods, can select the optimal step size at each iteration. The results indicate that, despite the proposed method and existing gradient methods having the same theoretical maximum convergence speed, the introduced variable step size mechanism in the proposed method consistently demonstrates superior convergence stability and performance when applied to practical problems.