Heterogeneous Cluster Application Communication Optimization and Computer Big Data Management

Abstract

In order to adapt to the constantly updated heterogeneous hardware and super-large-scale parallel computing environment and solve the problems of low programming level and difficult development, modification, and debugging of commonly used solutions, the author proposes a method for application communication optimization and computer big data management based on heterogeneous clusters. A new language mechanism is introduced to describe the multidimensional rule structure, arrangement, and communication mode of data and threads; and a software migration and optimization method between different types of heterogeneous systems based on the new language mechanism is proposed. And take direct method turbulence simulation as an example. Experimental results show that based on the Parray mechanism, it took only one week to complete the rapid migration of turbulence simulation applications on the Tianhe 1A system, and it was successfully run on a scale of 8192 cubic meters. Conclusion. The method realizes communication optimization and fast porting in different heterogeneous systems.