Erasure coding in windows azure storage

TLDR

Windows Azure Storage offers virtually unlimited, pay‑as‑you‑use cloud storage accessible from anywhere. The paper introduces Local Reconstruction Codes (LRC) for erasure coding. WAS employs erasure coding, specifically LRC, to achieve durable storage with low cost and consistently low read latencies. LRC lowers repair bandwidth and I/O, reduces the number of fragments read during reconstruction, and maintains low storage overhead compared to prior codes.

Abstract

Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere, at any time, and only pay for what they use and store. To provide durability for that data and to keep the cost of storage low, WAS uses erasure coding. In this paper we introduce a new set of codes for erasure coding called Local Reconstruction Codes (LRC). LRC reduces the number of erasure coding fragments that need to be read when reconstructing data fragments that are offline, while still keeping the storage overhead low. The important benefits of LRC are that it reduces the bandwidth and I/Os required for repair reads over prior codes, while still allowing a significant reduction in storage overhead. We describe how LRC is used in WAS to provide low overhead durable storage with consistently low read latencies.