2012 · 208 citations · 21 references
ComputingComputer EngineeringComputer ScienceCloud StorageFault-tolerant ComputingFault-tolerant SystemsCoding TheoryData RedundancyDistributed StorageError-correcting CodesFault-tolerant Storage SystemsStorage ReliabilityCode ConstructionsDistributed Storage CodesDistributed Storage ReliabilityHigh Availability StorageReed-solomon CodesReed-solomon CodingRegenerating CodesStorage Erasure Coding
Network coding for distributed storage promises higher storage efficiency, but designing simple, low‑cost exact‑repair codes with high data rates remains an open challenge, as prior explicit constructions are limited to a rate of 1/2. This paper introduces the first family of distributed storage codes that enable simple look‑up repair and achieve rates up to 2/3. The constructions implement exact repair by simple XORing of packets, making them easy to implement. Experimental evaluation in a realistic cloud storage simulator shows significant performance and reliability gains over replication and standard Reed‑Solomon codes.
Network codes designed specifically for distributed storage systems have the potential to provide dramatically higher storage efficiency for the same availability. One main challenge in the design of such codes is the exact repair problem: if a node storing encoded information fails, in order to maintain the same level of reliability we need to create encoded information at a new node. One of the main open problems in this emerging area has been the design of simple coding schemes that allow exact and low cost repair of failed nodes and have high data rates. In particular, all prior known explicit constructions have data rates bounded by 1/2. In this paper we introduce the first family of distributed storage codes that have simple look-up repair and can achieve rates up to 2/3. Our constructions are very simple to implement and perform exact repair by simple XORing of packets. We experimentally evaluate the proposed codes in a realistic cloud storage simulator and show significant benefits in both performance and reliability compared to replication and standard Reed-Solomon codes.
21
The Hadoop Distributed File System
Konstantin V. Shvachko, Hairong Kuang, Sanjay Radia et al. · 2010 · 4.8K citations
Network Coding for Distributed Storage Systems
Alexandros G. Dimakis, P. Brighten Godfrey, Yunnan Wu et al. · IEEE Transactions on Information Theory · 2010 · 2K citations
Sanjay Ghemawat, Howard Gobioff, Shun-Tak A. Leung · 2003 · 1.3K citations