Cutting the Tail: Designing High Performance Message Brokers to Reduce Tail Latencies in Stream Processing

Abstract

Over the last decade, organizations have become heavily reliant on providing near-instantaneous insights to the end user based on vast amounts of data collected from various sources in real-time. In order to accomplish this task, a stream processing pipeline is constructed, which in its most basic form, consists of a Stream Processing Engine (SPE) and a Message Broker (MB). The SPE is responsible for performing actual computations on the data and providing insights from it. MB, on the other hand, acts as an intermediate queue to which data is written by ephemeral sources and then fetched by the SPE to perform computations on. Due to the inherent real-time nature of such a pipeline, low latency is a highly desirable feature for them. Thus, several existing research works in the community focus on improving latency and throughput of the streaming pipeline. However, there is a dearth of studies optimizing the tail latencies of such pipelines. Moreover, the root cause of this high tail latency is still vague. In this paper, we propose a model-based approach to analyze in-depth the reasons behind high tail latency in streaming systems such as Apache Kafka. Having found the MB to be a major contributor of messages with high tail latencies in a streaming pipeline, we design and implement an RDMA-enhanced high-performance MB, called Frieda, with the higher goal of accelerating any arbitrary stream processing pipeline regardless of the SPE used. Our experiments show a reduction of up to 98% in 99.9th percentile latency for microbenchmarks and up to 31% for full-fledged stream processing pipeline constructed using Yahoo! Streaming Benchmark.