Capacity Planning and Headroom Analysis for Taming Database Replication Latency

Abstract

Internet companies like LinkedIn handle a large amount of incoming web traffic. Events generated in response to user input or actions are stored in a source database. These database events feature the typical characteristics of Big Data: high volume, high velocity and high variability. Database events are replicated to isolate source database and form a consistent view across data centers. Ensuring a low replication latency of database events is critical to business values. Given the inherent characteristics of Big Data, minimizing the replication latency is a challenging task.