Evaluation Platform for DDM Algorithms With the Usage of Non-Uniform Data Distribution Strategies

Abstract

Huge amounts of data are collected in numerous independent data storage facilities around the world. However, how the data is distributed between physical locations remains unspecified. Downloading all of the data for the purpose of processing it is undesirable and sometimes even impossible. Various methods have been proposed for performing data mining tasks, but the main problem is the lack of an objective strategy for comparing them. The authors present current research on a novel evaluation platform for distributed data mining (DDM) algorithms. The proposed platform opens up a new field to evaluate algorithms in terms of the quality of the results, transfer used, and speed, but also for the use of a non-uniform data distribution among independent nodes during algorithm evaluation. This work introduces a ‘data partitioning strategy’ term referring to a specific, not necessarily uniform data distribution. A brief evaluation for three clustering algorithms is also reported, showing the usability and simplicity of identifying differences in processing with the use of the platform.