Moirae: History-Enhanced Monitoring.

Abstract

In this paper, we present the benefits and challenges of integrating history into a near-real-time monitoring system. In particular, we address the specific problem of enhancing current events with historical information, by efficiently retrieving for each newly detected event similar past events (e.g., car accidents, network intrusions, server failures). We focus on applications where querying a historical log in its entirety would be too slow to meet application needs, and could potentially yield an overwhelming number of results. We propose a general purpose continuous monitoring system, called Moirae, that efficiently produces with each new event an approximate set of most similar recent events. We present the design of Moirae, show how our proposed architecture achieves the above goals, and discuss the applicability of the architecture to other types of integration between historical data and continuous monitoring. 1.