Performance Benchmarking of Different Methods to Solve Gauss-Newton Trust Region Subproblems

Abstract

Summary The Gauss-Newton (GN) trust region optimization methods perform robustly but may introduce significant overhead cost when using the conventional matrix factorization method to solve the associated GN trust region subproblem (GNTRS). Solving a GNTRS involves solving a nonlinear equation using an iterative Newton-Raphson (NR) method. In each NR iteration, a symmetric linear system can be solved by different matrix factorization methods, including Cholesky decomposition (CD), eigenvalue decomposition (EVD), and singular value decomposition (SVD). Because CD fails to factorize a singular symmetric matrix, we propose solving a GNTRS using the robust EVD method. In this paper, we analyze the performances of different methods to solve a GNTRS using different matrix factorization subroutines in LAPACK with different options and settings. The cost of solving a GNTRS mainly depends on the number of observed data (m) and the number of uncertainty parameters (n). When n≤m, we recommend directly solving the original GNTRS with n variables. When n>m, we propose an indirect method that transforms the original GNTRS with n variables to a new problem with m unknowns. The proposed indirect method can significantly reduce the computational cost by dimension reduction. However, dimension reduction may introduce numerical errors, which, in turn, may result in accuracy degradation and cause failure of convergence using the popular iterative NR method. To further improve the overall performance, we introduce a numerical error indicator to terminate the iterative NR process when numerical errors become dominant. Finally, we benchmarked the performances of different approaches on a set of testing problems with different settings. Our results confirm that the GNTRS solver using the EVD method together with the modified NR method performs the best, being both robust (no failure for all testing problems) and efficient (consuming comparable CPU time to other methods).