ZooKeeper: wait-free coordination for internet-scale systems

TLDR

ZooKeeper offers a wait‑free, event‑driven interface with FIFO guarantees and linearizability, enabling high‑performance coordination. ZooKeeper is presented as a simple, high‑performance coordination kernel for distributed applications, allowing clients to build more complex primitives. It combines group messaging, shared registers, and distributed lock services in a replicated centralized service, letting read requests be served locally for high throughput. Under read‑heavy workloads, ZooKeeper handles tens to hundreds of thousands of transactions per second, enabling extensive client use.

Abstract

In this paper, we describe ZooKeeper, a service for coordinating processes of distributed applications. Since ZooKeeper is part of critical infrastructure, ZooKeeper aims to provide a simple and high performance kernel for building more complex coordination primitives at the client. It incorporates elements from group messaging, shared registers, and distributed lock services in a replicated, centralized service. The interface exposed by Zoo-Keeper has the wait-free aspects of shared registers with an event-driven mechanism similar to cache invalidations of distributed file systems to provide a simple, yet powerful coordination service. The ZooKeeper interface enables a high-performance service implementation. In addition to the wait-free property, ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all requests that change the ZooKeeper state. These design decisions enable the implementation of a high performance processing pipeline with read requests being satisfied by local servers. We show for the target workloads, 2:1 to 100:1 read to write ratio, that ZooKeeper can handle tens to hundreds of thousands of transactions per second. This performance allows ZooKeeper to be used extensively by client applications.