One-Rank Linear Transformations and Fejer-Type Methods: An Overview

Abstract

Subgradient methods are frequently used for optimization problems. However, subgradient techniques are characterized by slow convergence for minimizing ravine convex functions. To accelerate subgradient methods, special linear non-orthogonal transformations of the original space are used. This paper provides an overview of these transformations based on Shor’s original idea. Two one-rank linear transformations of Euclidean space are considered. These simple transformations form the basis of variable metric methods for convex minimization that have a natural geometric interpretation in the transformed space. Along with the space transformation, a search direction and a corresponding step size must be defined. Subgradient Fejer-type methods are analyzed to minimize convex functions, and Polyak step size is used for problems with a known optimal objective value. Convergence theorems are provided together with the results of numerical experiments. Directions for future research are discussed.