Parallelizing I/O intensive applications for a workstation cluster

Abstract

An I/O intensive application, parallel full text retrieval based on a signature file method, is studied. The text retrieval system is implemented on a cluster of DEC5000 workstations connected by Ethernet. Experiments are performed to evaluate the benefit and cost for running such an application in a workstation cluster. Results show that substantial improvement in speed can be obtained through parallelism in disk accesses, despite the high communication and synchronization overhead that is incurred. Several factors that affect the performance of a parallel I/O application in this type of computing environment are discussed. The advantages of a workstation cluster are its large combined I/O buffer capacity, and possible concurrent accesses to disks local to each workstation. Our study demonstrates that these advantages, when exploited properly, can lead to effective performance improvement without the need for additional hardware.