Algorithms for descent along nodal straight lines in the problem of estimating regression equations using the least absolute deviations method

Abstract

A problem of estimating linear regression equations by the least absolute deviations method is considered. The exact methods of implementation of the method are significantly inferior in performance to the least square method. The fastest algorithm based on coordinate descent along nodal straight lines has a computational complexity proportional to the square of the number of observations, which limits the practical application of the method to monitoring and diagnostic tasks. The goal of the study is to describe a faster version of the descent along the nodal straight lines, as well as to evaluate the performance. Reduction of the computational costs is achieved due to the fact that instead of calculating the values of the objective function at nodal points, we find the derivative of the objective function in the vicinity of these points along the nodal line. The computational efficiency of gradient descent along nodal straight lines is estimated. For a typical computer, a comparative analysis of the average calculation time for various algorithms of descent along nodal straight lines is performed. A simple example is given to illustrate the implementation of a gradient descent procedure. Along with reduction of the computational costs, we also eliminated the possibility of accumulating computational errors when determining the values of the objective function for large samples. Moreover, gradient descent is quite simple for implementation. This makes it possible to use the method of least absolute deviations as an alternative to the least square method in various practical applications.