Rolling window time series prediction using MapReduce

Abstract

Prediction of time series data is an important application in many domains. Despite their advantages, traditional databases and MapReduce methodology are not ideally suited for this type of processing due to dependencies introduced by the sequential nature of time series. We present a novel framework to facilitate retrieval and rolling-window prediction of irregularly sampled large-scale time series data. By introducing a new index pool data structure, processing of time series can be efficiently parallelised. The proposed framework is implemented in R programming environment and utilises Hadoop to support parallelisation and fault tolerance. Experimental results indicate our proposed framework scales linearly up to 32-nodes.