Efficient dispersal of information for security, load balancing, and fault tolerance

Abstract

An Information Dispersal Algorithm (IDA) is developed that breaks a file F of length L = ↿ F ↾ into n pieces F i , l ≤ i ≤ n , each of length ↿ F i ↾ = L / m , so that every m pieces suffice for reconstructing F . Dispersal and reconstruction are computationally efficient. The sum of the lengths ↿ F i ↾ is ( n / m ) · L . Since n / m can be chosen to be close to l, the IDA is space efficient. IDA has numerous applications to secure and reliable storage of information in computer networks and even on single disks, to fault-tolerant and efficient transmission of information in networks, and to communications between processors in parallel computers. For the latter problem provably time-efficient and highly fault-tolerant routing on the n -cube is achieved, using just constant size buffers.