Serverless Big Data Processing using Matrix Multiplication as Example

Abstract

Serverless computing, or Function-as-a-Service (FaaS), is emerging as a popular alternative model to on-demand cloud computing. Function services are executed by a FaaS provider; a client no longer uses cloud infrastructure directly as in traditional cloud consumption. Is serverless computing a feasible and beneficial approach to big data processing, regarding performance, scalability, and cost effectiveness? In this paper, we explore this research question using matrix multiplication as example. We define requirements for the design of serverless big data applications, present a prototype for matrix multiplication using FaaS, and discuss and synthesize insights from results of extensive experimentation. We show that serverless big data processing can lower operational and infrastructure costs without compromising system qualities; serverless computing can even outperform cluster-based distributed compute frameworks regarding performance and scalability.