Scalable performance of the Panasas parallel file system

TLDR

The Panasas file system delivers scalable, fault‑tolerant, high‑performance distributed storage by using parallel, redundant access to object storage devices, per‑file RAID, distributed metadata, and client‑driven RAID that enable parallel data access and scalable RAID recovery across growing clusters. This study evaluates I/O, metadata, and recovery performance across Panasas clusters ranging from 10 to 120 storage nodes, 1 to 12 metadata nodes, and 1 to 100 compute clients. The evaluation is based on production deployments up to 500 storage nodes, 50 metadata managers, and 5,000 clients.

Abstract

The Panasas file system uses parallel and redundant access to object storage devices (OSDs), per-file RAID, distributed metadata management, consistent client caching, file locking services, and internal cluster management to provide a scalable, fault tolerant, high performance distributed file system. The clustered design of the storage system and the use of client-driven RAID provide scalable performance to many concurrent file system clients through parallel access to file data that is striped across OSD storage nodes. RAID recovery is performed in parallel by the cluster of metadata managers, and declustered data placement yields scalable RAID rebuild rates as the storage system grows larger. This paper presents performance measures of I/O, metadata, and recovery operations for storage clusters that range in size from 10 to 120 storage nodes, 1 to 12 metadata nodes, and with file system client counts ranging from 1 to 100 compute nodes. Production installations are as large as 500 storage nodes, 50 metadata managers, and 5000 clients.