High-order methods beyond the classical complexity bounds: inexact high-order proximal-point methods

Abstract

AbstractWe introduce a Bi-level OPTimization (BiOPT) framework for minimizing the sum of two convex functions, where one of them is smooth enough. The BiOPT framework offers three levels of freedom: (i) choosing the order p of the proximal term; (ii) designing an inexact pth-order proximal-point method in the upper level; (iii) solving the auxiliary problem with a lower-level non-Euclidean method in the lower level. We here regularize the objective by a $$(p+1)$$ ( p + 1 ) th-order proximal term (for arbitrary integer $$p\ge 1$$ p ≥ 1 ) and then develop the generic inexact high-order proximal-point scheme and its acceleration using the standard estimating sequence technique at the upper level. This follows at the lower level with solving the corresponding pth-order proximal auxiliary problem inexactly either by one iteration of the pth-order tensor method or by a lower-order non-Euclidean composite gradient scheme. Ultimately, it is shown that applying the accelerated inexact pth-order proximal-point method at the upper level and handling the auxiliary problem by the non-Euclidean composite gradient scheme lead to a 2q-order method with the convergence rate $${\mathcal {O}}(k^{-(p+1)})$$ O ( k - ( p + 1 ) ) (for $$q=\lfloor p/2\rfloor $$ q = ⌊ p / 2 ⌋ and the iteration counter k), which can result to a superfast method for some specific class of problems.