Proximal gradient methods with inexact oracle of degree q for composite optimization-Reference-Cited by-同舟云学术

Proximal gradient methods with inexact oracle of degree q for composite optimization

Published:2024-04-30 Issue: Volume: Page:
ISSN:1862-4472
Container-title:Optimization Letters
language:en
Short-container-title:Optim Lett

Author:

Nabou Yassine,Glineur François,Necoara Ion

Abstract

AbstractWe introduce the concept of inexact first-order oracle of degree q for a possibly nonconvex and nonsmooth function, which naturally appears in the context of approximate gradient, weak level of smoothness and other situations. Our definition is less conservative than those found in the existing literature, and it can be viewed as an interpolation between fully exact and the existing inexact first-order oracle definitions. We analyze the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems. We derive complexity estimates and study the dependence between the accuracy of the oracle and the desired accuracy of the gradient or of the objective function. Our results show that better rates can be obtained both theoretically and in numerical simulations when q is large.

Funder

H2020 Marie Skłodowska-Curie Actions

Unitatea Executiva pentru Finantarea Invatamantului Superior, a Cercetarii, Dezvoltarii si Inovarii

Publisher

Springer Science and Business Media LLC

Link

https://link.springer.com/content/pdf/10.1007/s11590-024-02118-9.pdf

Reference20 articles.

1. Agafonov, A., Kamzolov, D., Dvurechensky, P., Gasnikov, A., Takac, M.: Inexact tensor methods and their application to stochastic convex optimization. Optim. Methods Softw. (2017). https://doi.org/10.1080/10556788.2023.2261604

2. Beck, A.: First-Order Methods in Optimization. vol. 25, SIAM, (2017)

3. Bogolubsky, L., Gusev, G., Raigorodskii, A., Tikhonov, A., Zhukovskii, M., Dvurechensky, P., Gasnikov, A., Nesterov, Y.: Learning supervised PageRank with gradient-based and gradient-free optimization methods. In 30th Conference on Neural Information Processing Systems, (2016)

4. Bottou, L. Large-scale machine learning with stochastic gradient descent. In 19th International Conference on Computational Statistics, (2010)

5. Cohen, M.B., Diakonikolas, J., Orecchia, L.: On acceleration with noise-corrupted gradients. In International Conference on Machine Learning, (2018)