Adaptive online scheduling in storm

TLDR

The data‑driven economy demands efficient real‑time analytics, and Storm—a distributed computation system adopted by companies such as Twitter and Groupon—provides a disruptive platform for processing large data streams. The study focuses on how the deployment strategy of a Storm topology influences its performance. Storm schedules execution of topology components across available infrastructure, determining how operators and data flows are mapped to resources.

Abstract

Today we are witnessing a dramatic shift toward a data-driven economy, where the ability to efficiently and timely analyze huge amounts of data marks the difference between industrial success stories and catastrophic failures. In this scenario Storm, an open source distributed realtime computation system, represents a disruptive technology that is quickly gaining the favor of big players like Twitter and Groupon. A Storm application is modeled as a topology, i.e. a graph where nodes are operators and edges represent data flows among such operators. A key aspect in tuning Storm performance lies in the strategy used to deploy a topology, i.e. how Storm schedules the execution of each topology component on the available computing infrastructure.