Predictable Low-Latency Event Detection With Parallel Complex Event Processing

Abstract

The tremendous number of sensors and smart objects being deployed in the Internet of Things (IoT) pose the potential for IT systems to detect and react to live-situations. For using this hidden potential, complex event processing (CEP) systems offer means to efficiently detect event patterns (complex events) in the sensor streams and therefore, help in realizing a “distributed intelligence” in the IoT. With the increasing number of data sources and the increasing volume at which data is produced, parallelization of event detection is crucial to limit the time events need to be buffered before they actually can be processed. In this paper, we propose a pattern-sensitive partitioning model for data streams that is capable of achieving a high degree of parallelism in detecting event patterns, which formerly could only consistently be detected in a sequential manner or at a low parallelization degree. Moreover, we propose methods to dynamically adapt the parallelization degree to limit the buffering imposed on event detection in the presence of dynamic changes to the workload. Extensive evaluations of the system behavior show that the proposed partitioning model allows for a high degree of parallelism and that the proposed adaptation methods are able to meet a buffering limit for event detection under high and dynamic workloads.