Erasure Codes for Increasing the Availability of Grid Data Storage

Abstract

In this paper, we describe the design of a highlyavailable Grid data storage system. Increased availability is ensured by data redundancy and file striping. Redundant data is computed using Reed-Solomon (RS) codes. The level of availability can be chosen for each individual file. Storage overhead is minimal compared to traditional redundancy strategies based on complete replication. Our prototype uses existing Grid data management tools (GridFTP) for communication, and the RS encoding and decoding is embedded in the client software. Performance measurements in wide area network prove the efficiency of our design for high-availability and striped file transfers.