Running Large-Scale Applications on Cluster Grids

Abstract

Grid computing enables the enormous, but often poorly utilized, computing resources existing on the Internet to be harnessed in order to solve large-scale problems. However, if they want to solve the problems of configuration and performance optimization of their large-scale applications, programmers wishing to exploit such computing resources have to deal with highly variable communication delays, security threats, machine and network failures, and the distributed ownership of the computing resources. On the other hand, Parallel Virtual Machine (PVM) is still widely used to develop high performance parallel applications on high performance/cost ratio computing platforms, such as workstation clusters. However, workstations belonging to networked clusters cannot often be exploited since they are provided with private IP addresses, being hidden from the Internet by publicly addressable IP front-end machines. This paper presents an extension of PVM that enables such clustered machines hidden from the Internet to take part in a PVM computation as host machines. This way, programmers can exploit low-cost, networked computing resources, such as collections of clusters, widely available within departmental organizations, in a well-known programming environment, such as PVM, without having to tackle the problems affecting grid computing, which often ends up turning the development of parallel applications into a burdensome activity as well as penalizing application performance.