A METHODOLOGY FOR SCALING UP CLASSIFICATION ALGORITHMS

Abstract

Contrary to much of the research in machine learning where there is a concentration on problems with relatively small volume of data, one of the main challenges of the today's data mining systems is their ability to handle data that is substantially larger than available main memory on a single processor. In this paper, we present a distributed technique that consists in combining the partial results of different classifiers supplied, in parallel, with different subsets of data. It is, actually, a two-phase process. At first, a number of classifiers are trained, each with a different subset of the data. Then, the trained classifiers are used in the construction of a new training data set which has exactly the same format as, but is substantially smaller than, the initial one. The new data set is used to train the final classifier through an iterative process, which is guided by a threshold concerning the size of the data set and the achieved increase in accuracy. We present extensive empirical tests, which demonstrate that the proposed technique significantly reduces time complexity, usually at the expense of a lower accuracy, compared to a single classifier supplied with all the data.