Selecting a Predictor Weighting Method by Press

Abstract

An empirical method (PRESS) for examining and contrasting the cross-validated prediction accuracies of some popular algorithms for weighting predictor variables was advanced and examined. The weighting methods that were considered were ordinary least squares, ridge regression, regression on principal components, and regression on an equally weighted composite. PRESS was executed on several data sets having varied characteristics, with each of the weighting techniques obtaining the greatest accuracy under some conditions. The degree of advantage or disadvantage offered by these alternate weighting algorithms relative to ordinary least squares was considered. As it was not possible to determine a priori which weighting technique would be most accurate for a particular data set from theoretical knowledge or from simple sample data characteristics, the sample specific PRESS method was proffered as possibly most appropriate when the researcher wishes to select from among the several alternate predictor weighting algorithms in order to achieve maximum cross-validated prediction accuracy.