The Swarm scalable storage system

Abstract

Swarm is a storage system that provides scalable, reliable, and cost-effective data storage. Swarm is based on storage servers, rather than file servers; the storage servers are optimized for cost-performance and aggregated to provide high-performance data access. Swarm uses a striped log abstraction to store data on the storage servers. This abstraction simplifies storage allocation, improves file access performance, balances server loads, provides fault-tolerance through computed redundancy, and simplifies crash recovery. We have developed a Swarm prototype using a cluster of Linux-based personal computers as the storage servers and clients; the clients access the servers via the Swarm-based Sting file system. Our performance measurements show that a single Swarm client can write to two storage servers at 3.0 MB/s, while four clients can write to eight servers at 16.0 MB/s.