The Design of the Borealis Stream Processing Engine

TLDR

Borealis is a second‑generation distributed stream processing engine developed at Brandeis, Brown, and MIT that extends Aurora’s core functionality and Medusa’s distribution model to deliver advanced capabilities required by emerging stream processing applications. The paper outlines the basic design and functionality of Borealis. Borealis tackles dynamic query revision and specification changes with revision records, time travel, and control lines, and introduces a scalable QoS‑based optimization model for server and sensor networks along with a fault‑tolerance model offering flexible consistency‑availability trade‑offs. Real‑world application examples demonstrate the necessity of dynamically revising query results and modifying query specifications.

Abstract

Borealis is a second-generation distributed stream processing engine that is being developed at Brandeis University, Brown University, and MIT. Borealis inherits core stream processing functionality from Aurora [14] and distribution functionality from Medusa [51]. Borealis modifies and extends both systems in non-trivial and critical ways to provide advanced capabilities that are commonly required by newly-emerging stream processing applications. In this paper, we outline the basic design and functionality of Borealis. Through sample real-world applications, we motivate the need for dynamically revising query results and modifying query specifications. We then describe how Borealis addresses these challenges through an innovative set of features, including revision records, time travel, and control lines. Finally, we present a highly flexible and scalable QoS-based optimization model that operates across server and sensor networks and a new fault-tolerance model with flexible consistency-availability trade-offs.