RECOMMENDATION ALGORITHM USING DATA CLUSTERING

Abstract

Recommender systems play a vital role in the marketing of various goods and services. Despite the intensive growth of the theory of recommendation algorithms and a large number of their implementations, many issues remain unresolved; in particular, scalability, quality of recommendations in conditions of sparse data, and cold start. A modified collaborative filtering algorithm based on data clustering with the dynamic determination of the number of clusters and initial centroids has been developed. Data clustering is performed using the k-means method and is applied to group similar users aimed at increase of the quality of the recommendation results. The number of clusters is calculated dynamically using the silhouette method, the determination of the initial centroids is not random, but relies on the number of clusters. This approach increases the performance of the recommender system and increases the accuracy of recommendations since the search for recommendations will be carried out within one cluster where all elements are already similar. Recommendation algorithms are software-implemented for the movie recommendation system. The software implementation of various methods that allow the user to receive a recommendation for a movie meeting their preferences is carried out: a modified algorithm, memory and neighborhood-based collaborative filtering methods. The results obtained for input data of 100, 500 and 2500 users under typical conditions, data sparsity and cold start were analyzed. The modified algorithm shows the best results – from 35 to 80 percent of recommendations that meet the user's expectations. The drop in the quality of recommendations for the modified algorithm is less than 10 per cent when the number of users increases from 100 to 2500, which indicates a good level of scalability of the developed solution. In the case of sparse data (40 percent of information is missing), the quality of recommendations is 60 percent. A low quality (35 percent) of recommendations was obtained in the case of a cold start – this case needs further investigation. Constructed algorithms can be used in rating recommender systems with the ability to calculate averaged scores for certain attributes. The modified recommendation algorithm is not tied to this subject area and can be integrated into other software systems.