Abstract
AbstractWe propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. Using a suitable reformulation of the given problem, our method combines the inexact restoration approach for constrained optimization with the trust-region procedure and random models. Differently from other recent stochastic trust-region schemes, our proposed algorithm improves feasibility and optimality in a modular way. We provide the expected number of iterations for reaching a near-stationary point by imposing some probability accuracy requirements on random functions and gradients which are, in general, less stringent than the corresponding ones in literature. We validate the proposed algorithm on some nonconvex optimization problems arising in binary classification and regression, showing that it performs well in terms of cost and accuracy, and allows to reduce the burdensome tuning of the hyper-parameters involved.
Funder
Università degli Studi di Firenze
Publisher
Springer Science and Business Media LLC
Subject
Applied Mathematics,Computational Mathematics,Control and Optimization
Reference34 articles.
1. Bellavia, S., Bianconcini, T., Krejić, N., Morini, B.: Subsampled first-order optimization methods with applications in imaging. In: Chen, K., Schonlieb, C., Tai, X., Younces, L. (eds.) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Switzerland AG (2021)
2. Curtis, F.E., Scheinberg, K.: Optimization methods for supervised machine learning: from linear models to deep learning. Leading Developments from INFORMS Communities. INFORMS, pp. 89–114 (2017)
3. Bottou, L., Curtis, F.C., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Rev. 60(2), 223–311 (2018)
4. Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)
5. Gower, R.M., Schmidt, M., Bach, F., Richtarik, P.: Variance-reduced methods for machine learning. In Proceedings of the IEEE, vol. 108, pp. 1968–1983 (2020)
Cited by
6 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献