Affiliation:
1. Irkutsk State Transport University (ISTU)
Abstract
<p>Previously, the author managed to reduce the problem of constructing a quite interpretable linear regression, estimated using ordinary least squares method, to a mixed-integer 0-1 linear programming problem. In such models, the signs of the estimates correspond to the substantive meaning of the factors, the absolute contributions of the variables to the overall determination are significant, and the degree of multicollinearity is small. The optimal solution to the formulated problem can also be found by generating all subsets method. The purpose of this article is to conduct a comparative analysis of the effectiveness of these two approaches. To conduct computational experiments, 5 sets of real statistical data of various volumes were used. As a result, more than 550 different mixed-integer 0-1 problems were solved using the LPSolve package under different conditions. At the same time, the efficiency of solving similar problems using the generating all subsets method in the Gretl package was assessed. In all experiments, our proposed method turned out to be many times more effective than the generating all subsets method. The highest efficiency was achieved in solving the subset selection problem from 103 variables, solving each of which by generating all subsets would require estimating approximately 2103 (10.1 nonillion) models, which a conventional computer would not have been able to cope with in 1000 years. In LPSolve, each of these problems was solved in 32 – 191 seconds. The proposed method was able to process a large data sample containing 40 explanatory variables and 515,345 observations in an acceptable time, which confirms the independence of its effectiveness from the sample size. It has been revealed that tightening the requirements for multicollinearity and absolute contributions of variables in the linear constraints of the problem almost always reduces the speed of its solution.</p>
Publisher
Moscow State University of Psychology and Education
Subject
Polymers and Plastics,General Environmental Science
Reference20 articles.
1. Strizhov V.V., Krymova E.A. Metody vybora regressionnykh modeley [Methods for selecting regression models]. Moscow, Vychislitel'nyy tsentr im. A.A. Dorodnitsyna RAN, 2010. 60 p.
2. Miller A. Subset selection in regression. CRC Press, 2002.
3. Konno H., Yamamoto R. Choosing the best set of variables in regression analysis using integer programming, Journal of Global Optimization, 2009, vol. 44, pp. 273–282. DOI:10.1007/s10898-008-9323-9.
4. Miyashiro R., Takano Y. Mixed integer second-order cone programming formulations for variable selection in linear regression, European Journal of Operational Research, 2015, vol. 247, pp. 721–731. DOI:10.1016/j.ejor.2015.06.081.
5. Miyashiro R., Takano Y. Subset selection by Mallows’ Cp: A mixed integer programming approach, Expert Systems with Applications, 2015, vol. 42, pp. 325–331. DOI:10.1016/j. eswa.2014.07.056.