ComputingComputer ScienceComputer SystemsBig Data AnalyticsCluster ComputingServerless ComputingData PipelinesDistributed Data ProcessingServerless Application DesignServerless ArchitectureServerless InfrastructureServerless Platform ArchitectureBig Data ArchitectureEvent-driven Data PipelinesPay-as-you-go Cost ModelingPay-as-you-go PricingPay-as-you-go RulesServerless AnalyticsServerless Data ProcessingServerless Performance Optimization
Serverless architectures organized around loosely-coupled function invocations represent an emerging design for many applications. Recent work mostly focuses on user-facing products and event-driven processing pipelines. In this paper, we explore a completely different part of the application space and examine the feasibility of analytical processing on big data using a serverless architecture. We present Flint, a prototype Spark execution engine that takes advantage of AWS Lambda to provide a pure pay-as-you-go cost model. With Flint, a developer uses PySpark exactly as before, but without needing an actual Spark cluster. We describe the design, implementation, and performance of Flint, along with the challenges associated with serverless analytics.
4
Eric Jonas, Qifan Pu, Shivaram Venkataraman et al. · 2017 · 412 citations
Serverless Computing: Design, Implementation, and Performance
Garrett McGrath, Paul Brenner · 2017 · 360 citations
Neil Savage · Communications of the ACM · 2018 · 37 citations