Design of computer big data processing system based on genetic algorithm

Abstract

Abstract In recent years, people have witnessed the rapid growth of data, and big data has penetrated into every aspect of people's lives. If a big data processing system wants to extract the hidden value behind massive data, it is inseparable from the support of a large number of underlying infrastructure resources. However, the one-time expensive investment in the initial economy and the complexity of the later work of operation and maintenance hinder the use of some small and medium-sized enterprises. Based on this background, with the continuous development of computer technology, this paper constructs a large-scale data processing system that introduces genetic algorithms, making full use of the advantages of on-demand self-service and the elastic expansion of computer technology, shortening the time required for data processing and data analysis. life cycle, so that more and more enterprises and organizations can start using big data processing technology. For fragmented big data obtained from different data sources, this paper adopts load balancing technology to provide horizontal service cluster scalability, and designs a separate system module for routine testing. The experimental results show that the designed function of the system can be realized, and the actual error is always lower than the specified error limit. It is hoped that the research work in this paper can provide useful reference and help for the design of computer big data processing system. This paper designs a kind of effective big data processing system by studying genetic algorithm and computer technology.