Development of an Enhanced C4.5 Decision Tree Algorithm Using a Memoized Mapreduce Model

Abstract

Abstract Classification technique in data mining focuses on prediction which is done by classical C4.5 decision tree algorithm, but limited by its computation complexities due to large datasets. However, this results to inefficient implementation of the algorithm with reference to computing time, memory utilization and data complexity. Meanwhile, several researches have been done to curb these limitations. One of such improvements is the parallelizing of the algorithm using the MapReduce model. This involves splitting the large dataset into smaller units and distributing them on multiple computers for parallel processing, but the recursive nature of the algorithm makes the computational cost high due to large number of calculations that are repeated. This research is aimed at further reducing computation time, using memoized MapReduce model that involves storing the result of previous calculations in a cache. Thus, when same calculations re-occur, the cached result is returned, thereby eliminating re-computation.