Lipschitz global optimization and machine learning: helping each other to solve complex problems

Abstract

In this paper we consider global optimization problems and methods for solving them. The numerical solution of this class of problems is computationally challenging. The most complex problems are multicriteria problems in which the objective functions are multiextremal and non-differentiable, and, moreover, given in the form of a “black box”, i.e. calculating the objective function at a point is a time-consuming operation. Particularly, we consider an approach to acceleration of the global search using machine learning methods. At the same time, the problem of tuning the hyperparameters of the machine learning methods themselves is very important. The quality of machine learning methods is substantially affected by their hyperparameters, while the evaluation of the quality metrics is a time-consuming operation. We also consider an approach to hyperparameter tuning based on the Lipschitz global optimization. These approaches are implemented in the iOpt open-source framework of intelligent optimization methods.