Ceph: a scalable, high-performance distributed file system

TLDR

Ceph separates data and metadata by using a CRUSH algorithm for data placement, delegating replication, failure detection, and recovery to autonomous OSDs, and employing a dynamic distributed metadata cluster for efficient, workload‑adaptable metadata management. Benchmarks demonstrate that Ceph delivers excellent I/O performance and scalable metadata handling, achieving over 250,000 metadata operations per second across diverse workloads.

Abstract

We have developed Ceph, a distributed file system that provides excellent performance, reliability, and scalability. Ceph maximizes the separation between data and metadata management by replacing allocation tables with a pseudo-random data distribution function (CRUSH) designed for heterogeneous and dynamic clusters of unreliable object storage devices (OSDs). We leverage device intelligence by distributing data replication, failure detection and recovery to semi-autonomous OSDs running a specialized local object file system. A dynamic distributed metadata cluster provides extremely efficient metadata management and seamlessly adapts to a wide range of general purpose and scientific computing file system workloads. Performance measurements under a variety of workloads show that Ceph has excellent I/O performance and scalable metadata management, supporting more than 250,000 metadata operations per second.