Fuzzy clustering in high-dimensional spaces: Visualization and performance metrics

Abstract

In general, most of the clustering algorithms deal with high-dimensional data. Therefore, the resulting clusters are high-dimensional geometrical objects, which are difficult to analyse, visualize and interpret. Principal component analysis (PCA) is a widely used classical technique for unsupervised dimensional reduction of datasets which extracts smaller uncorrelated data from the original high dimensional data space. The methodology of classical PCA is based on orthogonal projection defined in convex vector space. Thus, a norm between two projected vectors is unavoidably smaller than the norm between any two objects before implementation of PCA. Due to this, in some cases when the PCA cannot capture the data structure, its implementation does not necessarily confirm the real similarity of data in the higher dimensional space making the results unacceptable. Furthermore, when applied with fuzzy clustering algorithms, it demonstrates its weakness in capturing the data structure what affects the PCA mapping quality. In this research we propose a new fuzzy PCA algorithm (FuzPCA) combining FCM and PCA which builds not only fuzzy similarity measures based on the membership functions calculated by FCM but also a measure corresponding to the rate of dissimilarities between the clustering structures of the dataset objects and their dimensions. We formulated the respective new fuzzy covariance matrix which is processed by the PCA during subsequent iterations. FuzPCA demonstrates higher clustering discriminatory ability and visualization accuracy than regular FCM when applied to high-dimensional datasets, in cases when clusters’ sizes or densities are different as well as in discovering small clusters. The function integrated with FuzPCA Silhouette provides preliminary information and visualization allowing faster choice of number of clusters which is additionally refined after implementation of fuzzy clustering validation metrics. The projected 2- and 3-D topologies of FuzPCA are more reliable, significantly improve the visualization results and demonstrate high mapping quality.