Database System Based on 3Dmax Big Data Mining Technology

Abstract

INTRODUCTION: This project intends to study the mining method of FP-growth frequent items in 3Dmax big data under the Hadoop framework and combined with the Map Reduce development model. Firstly, the transaction database is selected according to the frequency of each transaction and the corresponding projection library is generated. Then the obtained image database is distributed on each node computer. Then, under the guidance of the node machine, the projection is divided into different regions to produce several smaller sub-databases. The method is parallelized by using node machine to generate local frequency items. Finally, all the local frequency sets are merged into one complete frequency set. This method does not need to generate as many FP trees as the regular FP-growth method. This method can overcome the computational failure problem caused by the limited memory of a single computer by the conventional FP-Growth method and other methods. At the same time, because the sublibraries of partitions are similar in size, the load distributed to each node machine is more balanced. The effectiveness of the algorithm is improved.