The convergence of the Ben-Israel iteration for nonlinear least squares problems

Abstract

Ben-Israel [1] proposed a method for the solution of the nonlinear least squares problem min x ∈ D ‖ F ( x ) ‖ 2 {\min _{x \in D}}{\left \| {F(x)} \right \|_2} where F : D ⊂ R n → R m F:D \subset {R^n} \to {R^m} . This procedure takes the form x k + 1 = x k − F ′ ( x k ) + F ( x k ) {x_{k + 1}} = {x_k} - F’{({x_k})^ + }F({x_k}) where F ′ ( x k ) + F’{({x_k})^ + } denotes the Moore-Penrose generalized inverse of the Fréchet derivative of F. We give a general convergence theorem for the method based on Lyapunov stability theory for ordinary difference equations. In the case where there is a connected set of solution points, it is often of interest to determine the minimum norm least squares solution. We show that the Ben-Israel iteration has no predisposition toward the minimum norm solution, but that any limit point of the sequence generated by the Ben-Israel iteration is a least squares solution.