Towards an architecture for big data analytics leveraging edge/fog paradigms

Abstract

An industry transformation is being boosted by Big Data and Cloud technologies. We present a Big Data architecture, which expands the life cycle of data processing through the Edge, Fog and Cloud computing layers. The proposed architecture takes advantage of the strengths of each: the Cloud layer executes heavy analytical processes, the Fog is responsible for the ingestion and performing aggregations, and the Edge manages devices and actuators. The proposed architecture tackles two main goals, 1) latencies and response times can be reduced by bringing the analytics closer to where the data is generated and 2) the use of computing resources is optimised. In order to conceptualise this architecture, an orchestration module is proposed with the goal of optimising the deployment of analytical workloads across the three layers, by evaluating their computing resources. In addition to this, another module is designed to monitor the performance of such workloads allowing the redistribution of tasks assigned to each node. These modules will be implemented in a real case scenario in the train domain.