2021 · 35 citations · 53 references
ComputingComputer ScienceComputer SystemsCloud ArchitectureCloud InfrastructureCloud Performance OptimizationCloud Resource ManagementCost OptimizationCost-performance TradeoffsLatency OptimizationResource ProvisioningResource SchedulingEnd-to-end LatencyGradient DescentMicroservice DeploymentMicroservices DeploymentMicroservices ScalabilityService DeploymentService LevelService Level Objective Optimization
Modern cloud services are implemented as graphs of loosely-coupled microservices to improve programmability, reliability, and scalability. Service Level Objectives (SLOs) define end-to-end latency targets for the entire service to ensure user satisfaction. In such environments, each microservice is independently deployed and (auto-)scaled. However, it is unclear how to optimally scale individual microservices when end-to-end SLOs are violated or underutilized, and how to size each microservice to meet the end-to-end SLO at minimal total cost. In this paper, we propose Parslo---a Gradient Descent-based approach to assign partial SLOs among nodes in a microservice graph under an end-to-end latency SLO. At a high level, the Parslo algorithm breaks the end-to-end SLO budget into small incremental "SLO units", and iteratively allocates one marginal SLO unit to the best candidate microservice to achieve the highest total cost savings until the entire end-to-end SLO budget is exhausted. Parslo achieves a near-optimal solution, seeking to minimize the total cost for the entire service deployment, and is applicable to general microservice graphs that comprise patterns like dynamic branching, parallel fan-out, and microservice dependencies. Parslo reduces service deployment costs by more than 6x in real microservice-based applications, compared to a state-of-the-art partial SLO assignment scheme.
53
Luiz André Barroso · Communications of the ACM · 2013 · 1.7K citations
Christina Delimitrou, Christos Kozyrakis · 2013 · 618 citations
Jason Mars, Lingjia Tang, Robert Hundt et al. · 2011 · 604 citations